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PREFACE 


This  handbook,  in  conjunction  with  a  revised  AR  37-18  (Weapons/ 
Support  Systems  Cost  Categories  and  Elements),  is  published  in 
response  to  a  need  expressed  in  the  Weapon  System  Acquisition 
Improvement  Program  to  provide  a  vehicle  for  developing  a  standard 
cost  methodology.  The  handbook  attempts  to  fulfill  part  of  this 
objective  by  providing  a  better  understanding  of  the  cost  estimating/ 
analysis  process,  by  promoting  standardization  in  cost  analyses, 
and  by  contributing  to  a  better  cost  analysis  capability  in  the 
Army.  The  revised  AR  37-18  provides  more  detailed  guidance  and 
standardization  requirements.  The  handbook  further  contributes 
another  step  toward  attaining  the  objectives  of  the  Army  Cost 
Analysis  Program  as  stated  in  AR  11-18. 
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PART  I  -  THE  PROCESS  OF  COST  ESTIMATING 


i 


c 


SUMMARY 


T  i 

This-  handbook  is  a  compilation  of  subjects  covering  the  spectrum 
of  cost  estimating  and  cost  analysis.  The  subjects  are  broken  down 
into  two  major  parts:  (tO  the  process  of  cost  estimating  and— (-2) 
-other-  special  considerations  which  affect  the  cost  estimating 
process.  Three  appendices  covering  subjects  of  general  interest 
are  also  included.  < 

The  analyst  must  first  realize  that  cost  analysis  emphasizes  the 
systems  approach.  An  individual  development  or  production  estimate, 
for  example,  is  part  of  a  larger  system  **  the  Life  Cycle  Cost 
Estimate  (LCCE).  The  LCCE  of  a  material  system  then  becomes  part  of 
an  even  larger  system  -  the  management  system  for  development  of  Army 
systems.  The  analyst  must  understand  the  system  within  which  he 
labors  and  the  effects  on  cost  estimates  of  the  dynamic  nature  of  that 
system. 

At  a  more  specific  level  it  is  acknowledged  that  cost  estimating 
requires  a  myriad  of  information.  This  Includes  information  relating 
to  the  physical  and  performance  characteristics  of  the  system  under 
study,  management  information  relating  to  the  system,  and  historical 
information  relating  to  comparable  systems.  Emphasis  is  placed  on 
the  major  sources  of  information  and  the  configuration  of  that 
information. 

At  the  heart  of  cost  analysis  are  those  techniques  by  which  cost 
estimates  are  made.  These  are:  the  parametric  approach;  the 
analogy  approach;  the  engineering  approach,  and  expert  opinion. 

Of  the  four,  the  parametric  approach  is  considered  most  useful  and 
receives  heaviest  emphasis.  The  use  of  a  specific  approach  varies 
with  the  amount  and  reliability  of  data  available  and  each  approach 
has  limitations. 

Computation  of  the  estimate  does  not  complete  the  job  for  the  cost 
analyst.  Invariably  there  are  uncertainties  in  the  estimate  which 
must  be  itemized,  quantified,  and  evaluated.  Additionally,  the 
sensitivity  of  the  estimate  to  possible  fluctuations  of  input 
parameters  must  also  be  quantified  and  evaluated  if  the  estimate  is 
to  be  of  use  to  decision  makers.  Some  common  means  of  expressing 
these  aspects  of  cost  estimating  are  presented.  Appendix  A  emphasizes 
the  development  and  the  use  of  mathematical  models  in  the  estimating 
process  and  in  evaluating  the  estimate  validity  and  sensitivity. 
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An  analysis  must  further  weigh  economic  considerations  as  they 
impinge  on  the  cost  estimate.  The  effects  of  the  present  value  of 
money  and  inflation  can  significantly  change  the  perspective  in 
making  trade-offs  among  projects  which  have  differing  expenditure 
and  cost/benefit  patterns  over  time. 

Conducting  an  analysis  or  making  an  estimate  is  only  part  of  the 
task  for  an  analyst.  The  study  must  be  presented  in  a  form  which  is 
understandable  and  usable  by  the  individual  who  must  make  decisions 
based  on  the  study.  The  presented  study  must  be  fully  documented, 
the  assumptions  clearly  stated  and  the  alternatives  properly  considered. 
Presentation  guidelines  and  formats  are  offered  to  assist  both  the 
analyst  and  the  decision  maker. 

Of  further  use  by  the  analyst  at  particular  points  in  the  system 
life  cycle  are  those  means  by  which  cost  growth  is  defined  and 
measured,  by  which  cost  effectiveness  analyses  are  conducted  and 
by  wh?ch  controls  may  be  established.  In  the  latter  case  a 
relatively  new  concept  for  evaluating  cost  estimates,  derived 
from  a  Study  of  Trends  and  Escalation  of  Costs  (STECPLOT),  should 
be  considered  by  the  analyst  as  a  potential  control  device. 

It  is  these  aspects  of  cost  analysis  which  are  presented  in  detail 
in  this  handbook  in  support  of  the  Weapon  System  Acquisition 
Improvement  Program  and  in  pursuit  of  the  Army  Cost  Analysis 
Program  Objectives. 


CHAPTER  1 


THE  LIFE  CYCLE  AND  DECISION  POINTS 
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As  a  system  progresses  from  the  concept  phase  to  full  deployment, 
increasingly  complex  decisions  must  be  made  which  commit  the  Army  to 
greater  expenditures  of  resources.  Decisions  can  be  more  competently 
made  if,  at  each  significant  stage,  the  decision-maker  is  provided  an 
accurate  estimate  of  all  the  costs  associated  with  that  system  from 
birth  to  death.  Estimates  of  this  type  are  called  Life  Cycle  Cost 
Estimates  (LCCE). 

Realistic  Life  Cycle  Cost  Estimates  should  be  provided  as  early  as 
possible  in  the  life  cycle.  It  is  recognized  that  in  early  system 
stages  little  information  may  be  available  from  which  to  derive  a 
complete  estimate.  Still  decisions  must  be  made  early  in  the  life 
cycle  and  therefore  estimates  should  be  made  as  early  as  possible  and 
updated  continuously  as  the  system  progresses  and  more  data  is  made 
available.  Parametric  estimating  techniques  are  recommended  for  early 
estimates  when  little  information  is  available  (se,e  chapter  on  Techniques 
of  Costing).  Later  in  the  life  cycle  the  use  of  other  techniques  are 
made  possible  by  the  availability  of  more  firm  data. 

To  develop  a  LCCE,  the  cost  analyst  must  consider  all  costs  from  research 
and  development  through  investment  arid  operation  of  the  system.  DA 
Pamphlet  11-2:5,  Life  Cycle  Management  Model  for  Army  Systems,!./  describes 
a  Life  Cycle  in  a  manner  different  from  a  simple  accumulation  of  costs 
and,  since  many  of  the  management  documents  are  related  to  that 
description,  it  is  necessary  for  the  analyst  to  understand  it.  The 
model  presented  is  a  guide  however,  and  must  be  used  with  the  under¬ 
standing  that  management  of  the  development  of  Army  systems  is  a  dynamic 
process.  Improvements  are  constantly  being  made  and  documents  describing 
management  systems  are  often  quickly  outdated  in  some  aspects. 

DA  Pamphlet  11-25  envisions  a  system  as  progressing  through  the  following 
stages: 

Concept  Formulation 

Contract  Definition 

Development  and  Production 

Operations  and  Disposal 


1/  At  this  writing  (March  1971),  a  major  revision  of  DA  Pam  11-25  is 
being  completed.  Its  purpose  is  to  reduce  the  number  of  actions  required 
in  the  complete  life  of  a  system  and  to  reduce  the  paperwork  involved  by 
consolidation  of  all  reports  requirements  of  the  Life  Cycle  Cost  Model 
(e.g,,  QMA,  QMR,  SDP,  ADP,  EDPS)  into  a  single  "materiel  need’1  document 
that  will  follow  an  item  through  the  system  incorporating  greater  levels 
of  detail  as  more  information  about  the  progress  of  the  item  becomes  known. 
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The  Pamphlet  presents,  within  the  outline  of  these  four  phases,  a  complete 
Life  Cycle  Flow  Chart  of  240  actions  affecting  a  system  along  with  a 
narrative  to  describe  the  model. 

The  Life  Cycle  Cost  Estimating  process  is  a  continuous  cycle.  If  progress 
in  the  development  of  a  system  and  increased  data  availability  do  not  make 
an  update  necessary,  then  the  passage  of  time  and  economic  changes  will. 

Thus,  the  original  cost  estimate  must  be  used  only  as  an  initial  base  and 
from  this  base  revised  estimates  must  be  made  as  the  life  cycle  progresses. 
The  cost  estimate  must  be  seen  as  an  evolving  base  which  can  be  reasonably 
tracked  from  the  original  base  to  the  current  estimate.  It  is  emphasized 
that  estimates  must  be  reevaluated  at  each  significant  point  of  development. 

Of  interest  to  the  cost  analyst  are  those  actions  affecting  a  system  during 
its  life  cycle  at  which  decisions  are  made  or  inputs  provided  which  will 
result  in  information  to  improve  the  life  cycle  cost  estimate  or  a  need  to 
change  it.  Also  of  interest  are  those  points  in  the  life  cycle  at  which 
cost  estimates  are  used  in  the  decision  making  process.  The  remainder  of 
this  chapter  then  is  concerned  not  with  the  entire  life  cycle  but  with  some 
of  the  major  points  in  the  life  cycle  which  concern  the  cost  analyst.  The 
format  used  will  be  to  indicate  the  action  taken  and  the  cost  analysis  impact 
of  that  action.  The  actions  will  be  keyed  to  the  life  cycle  flow  chart  shown 
in  DA  Pam  11-25. 

CONCEPT  FORMULATION  STAGE 

Block  24;  Technical  Approach  Identified 

Cost  Analysis  (CA)  Impact:  When  the  technical  approach  to  materiel  develop¬ 
ment  has  been  identified  a  Qualitative  Material  Approach  (QMA)  is  produced. 
The  QMA  better  defines  problem  ^solutions  and  provides  the  cost  analyst 
with  additional  information  for  development  of  the  Life  Cycle  Cost  Estimate 
(LCCE)  to  include  physical  characteristics,  approximate  time  to  develop  and 
procure,  personnel  implications  and  technical  risk. 

Block  28:  Advanced  Development  Plan  (ADP) 

CA  Impact:  The  materiel  developer  must  provide  cost  estimates  for  the 
development  plans.  These  estimates  are  then  reviewed '-and  analyzed  by  DA 
or  DOD.  If  approved,  the  plans  commit  the  Army  to  further  expenditures  of 
funds.  Therefore,  the  cost  analyst  must  be  aware  of  all  the  known  factors 
impacting  on  the  system  under  study  to  provide  the  best  estimate  at  this 
time. 
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Block  33: 


Analysis  of  Trade-Offs  -  Trade-Off  Evaluations. 


CA  Impact:  Trade-off  analysis  is  required  prior  to  contract  definition. 
These  tradeoffs,  produced  by  the  material  developer,  must  include  the 
costs  associated  with  the  alternatives  to  assist  in  decision  making. 

Lack  of  sufficient  data  at  this  time  may  necessitate  a  reliance  on 
parametric  or  statistical  analysis  techniques  by  the  cost  analyst. 

Block  36:  Approval  of  Unit  Structure  and  Tentative  Basis  of  Issue  Plan 
(BOI). 

CA  Impact:  Approved  plans,  furnished  by  HQ  DA  (Assistant  Chief  of  Staff 
for  Force  Development  (ACSFOR))  provide  the  cost  analyst  with  more  firm 
guidance  and  additional  data  for  the  LCCE  concerning  the  plan  for  the 
units  which  will  man  the  material  being  developed  and  the  plan  for 
issuing  the  material. 

Block  39:  Coat  Effectiveness  Analysis. 

CA  Impact:  Here  the  system  is  evaluated  against  competing  systems 
using  cost  and  effectiveness  as  governing  criteria.  Accurate  cost 
estimates  provided  by  the  analyst  are  an  essential  element  in  this  step. 

Blocks  41,  42:  Approval  of  Qualitative  Material  Requirements  (QMR). 

CA  Impact:  The  existing  LCCE  is  used  as  a  major  input  in  evaluating  ♦'he 
proposed  QMR.  If  changes  are  made  to  the  proposed  QMR,  cost  estimates 
must  be  changed  accordingly.  Approval  of  the  QMR  normally  includes  a 
directive  to  the  developing  agency  to  begin  an  end  item  development 
project. 

Block  48:  Develop  Preliminary  System  Development  Plan  (SDP). 

CA  Impact:  The  preliminary  SDP  provides  the  cost  analyst  with  a 
valuable  input  in  the  form  of  a  consolidation  of  the  plans  for  project 
development.  As  far  as  possible  at  the  time,  it  will  include  aspects 
of  operational,  maintenance  support,  product  assurance  test,  personnel 
and  training,  facility,  financial,  installation  and  checkout,  procure¬ 
ment  and  production  plans.  A  major  effort  for  the  cost  analyst  is  to 
convert  information  from  the  SDP  into  the  Life  Cycle  Cost  Estimate. 

CONTRACT  DEFINITION  PHASE 

Block  74:  Develop  Proposed  Support,  Procurement  and  Management  Plan. 

CA  Impact:  The  preliminary  SDP  (Block  48)  is  updated  based  on  engineering 
accomplished  and  other  refinements  in  system  development.  The  updated 
plans  must  be  scrutinized  by  the  cost  analyst  and  the  LCCE  updated 
accordingly. 
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Block  82:  HQ  DA  Evaluation  of  Program  Status. 


CA  Impact:  This  is  one  of  several  times  during  the  life  cycle  at  which  the 
system  is  evaluated  by  DA.  In  fact,  a  system  evaluation  must  be  made  and 
a  decision  to  proceed  must  be  accomplished  before  the  system  can  enter  a 
new  life  cycle  phase.  Costs  associated  with  the  system  constitute  an 
Important  decision  criteria.  Any  program,  changes  made  as  a  result  of  the 
evaluations  must  be  taken  into  account  by  the  analyst  in  updating  the  LCCE. 

Block  88;  Review  of  Source  Selection  Evaluation  Board  (SSEB)  Appraisal  and 
Recommended  Development  Approach  and  Developer. 

CA  Impact:  The  recommendation  from  which  source  the  system  will  be  procured 
inherently  considers  the  costs  involved.  The  source  recommendation  will 
allow  the  analyst  to  reduce  the  number  of  estimates  to  only  those  in 
contention. 

DEVELOPMENT  AND  PRODUCTION  PHASE 
Block  105:  Development  Contract. 

CA  Impact:  The  contract  for  full  scale  development  is  placed  into  negotiation 
and  program  cost  estimates  are  used  to  evaluate  the  contractors  proposal. 

Early  production  items  which  require  funds  will  also  require  cost  estimates. 
The  type  of  cost  information  reports  required  from  the  contractor  will  also 
be  rr.-.Je  firm.  These  will  later  be  a  valuable  source  of  information  for  the 
cost  analyst. 

Block  112:  Prepare  Total  Maintenance  Requirements  by  Location. 

CA  Impact:  This  action,  accomplished' by  the  Project  Manager,  considers 
the  full  range  of  maintenance  support  which  will  be  required.  The  cost 
analyst  must  consider  the  same  requirements  in  order  to  develop  cost  trade¬ 
off  studies  used  to  provide  information  relating  to  preferred  site  locations. 
Additionally,  the  cost  of  maintenance  requirements  will  be  a  significant 
element  of  the  operating  cost  portion  of  the  LCCE. 

Block  132:  Procurement  of  Equipment  and  Missiles,  Army  (PEMA)  and  Military 
Construction,  Army  (MCA)  Program  Elements. 

CA  Impact:  Staffing  of  Program  Change  Requests  for  PEMA  and  MCA  funds 
results  in  PEMA  and  MCA  data  which  must  be  incorporated  into  the  Five  Year 
Defense  Program  (FYDP).  Analyst  estimates  of  PEMA  and  MCA  are  instrumental 
as  inputs  for  management  decisions  relating  to  approval  of  these  funds. 


Blocks  159,  160:  Prepare  Draft  Army  Materiel  Plan  (AMP)  and  Equip¬ 
ment  Distribution  and  Planning  Study  (EDPS). 


GA  Impact:  Analysts  estimates  of  procurement  costs  play  a  major  role 
in  the  development  of  the  AMP  which  is  a  consolidation  of  plans  for 
procurement. 

Block  186;  Approval  of  Complete  Basis  of  Issue  Plan  (BOI). 

CA  Impact;  Approval  of  the  complete  BOI  provides  the  analyst  more 
firm  guidance  for  updating  the  LCCE.  More  accurate  information  can 
now  be  determined  with  regard  to  the  funds  necessary  to  support  the 
program,  prepare  the  budget,  and  update  the  AMP. 

OPERATION  AND  DISPOSAL  PHASE 

Block  212;  Follow-On  Production. 

CA  Impact:  After  initial  production  to  satisfy  immediate  needs,  further 
production  may  be  required  either  by  the  initial  contractor  or  additional 
producers.  The  analyst  may  be  required  to  project  future  costs  based 
on  historical  data  through  use  of  parametric  analysis  or  cost-quantity 
relationships. 

Block  227:  Recommend  Reduction/Elimination  of  Equipment  Item(s)  from 
Supply  System. 

CA  Impact;  At  a  certain  point  in  time  it  can  be  expected  that  a  system 
could  be  reduced,  replaced,  or  eliminated  at  an  economic  advantage  or 
to  increase  effectiveness.  Here  the  analyst  may  have  to  employ  the 
techniques  of  economic  analysis  to  assist  in  determining  the  best 
course  of  action. 

Concentration  on- these  life  cycle  points  is  not  to  imply  that  the  cost 
analyst  should  be  aware  only  of  those  actions  listed.  Certainly  there 
are  many  other  points  in  the  life  cycle  at  which  changes  to  the  system 
are  made.  Therefore,  the  analyst  must  be  aware  of  the  current  status 
at  any  time  and  he  must  also  anticipate  any  possible  changes  to  the 
system  in  question  so  that  he  will  be  able  to  provide  the  best  possible 
LCCE  at  any  time. 

To  be  effective,  the  cost  estimates  must  be  as  refined  and  up  to  date 
as  possible.  The  analyst  must  therefore  have  a  good  grasp  of  such 
factors  as  production  schedules,  appropriation  action,  and  the 
production  engineering  status. 

The  process  of  Life  Cycle  Cost  Estimating  must  be  thoroughly  documented 
to  ensure  its  completeness,  continuity,  and  consistency  with  earlier 
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estimates.  (See  chapter  on  Presentation  of  the  Estimate  for  methods.)  An 
audit  trail  must  be  clearly  traceable  back  to  the  sources  or  basis  of  cost 
inputs  used.  This  permits  an  evaluation  of  sources  as  well  as  of  estimated 
costs.  To  aid  in  this  evaluation,  all  areas  where  judgment  has  been  used 
must  be  identified.  Significant  changes  in  costs  as  the  life  cycle  pro¬ 
gresses  must  also  be  identified  and  documented  and  reasons  for  these 
changes  must  be  determined.  (See  chapter  on  Cost  Growth  for  a  discussion 
of  reasons  for  changes  in  costs.) 

Documentation  of  estimates  will  further  allow  a  reviewer  to  track  from  the 
first  estimate  to  the  last  and  from  estimates  to  actual  cost  data.  Proper 
documentation  could  ultimately  lead  to  the  establishment  of  a  useful  data 
base  for  follow-on  studies. 


CHAPTER  2 


PRELIMINARY  GUIDANCE 


Preliminary  guidance  is  that  guidance  provided  to  initiate  a  costing 
effort.  It  should  be  provided  in  conjunction  with  the  tasking  document. 
Its  purpose  is  to  provide  the  estimator  with  sufficient  information 
to  take  the  first  steps  toward  preparation  of  the  estimate.  These 
first  steps  would  normally  consist  of  developing  a  plan  for  preparation 
of  the  estimate,  conducting  a  data  search,  and  selection  of  a  model. 
Failure  to  provide  adequate  preliminary  guidance  can  result  in  lost 
time  and  wasted  effort,  and  frequently  leads  to  a  less  than  satis¬ 
factory  product.'* 

As  a  starting  point  for  the  estimate,  the  estimator  must  be  provided 
with  certain  basic  information.  In  general,  this  basic  information 
consists  of: 

a.  What  type  of  estimate  is  required?  To  what  level  of  detail? 

Is  the  system  adequately  defined?  What  is  the  intended  use  of  the 
estimate?  At  what  point  in  the  life  cycle  is  the  system?  Answers 
to  these  questions  provide  the  estimator  with  a  feel  for  what  type 
of  data  might  be  available  and  what  type  of  end  result  his  analysis 
is  to  produce.  For  an  item  in  concept  formulation,  a  parametric 
estimate  of  production  cost  for  an  estimated  quantity  might  be  all 
that  is  possible;  for  an  item  in  production,  a  much  greater  level  of 
detail  is  available.  Likewise,  the  intended  use  of  the  estimate 
will  indicate  what  costs  are  to  be  included.  An  estimate  to  support 
a  program  estimate  differs  from  one  that  portrays  total  economic 
impact . 


b.  How  much  time  is  allowed  for  preparation  of  the  estimate? 

If  time  available  Impacts  on  the  accuracy  of  the  estimate,  is  the 
tasking  organization  aware  of  this?  The  organization  requesting  the 
estimate  may  be  seeking  a  "quick  and  dirty"  estimate  for  limited  use; 
on  the  other  hand,  it  may  be  seeking  a  well-documented  figure  to 
support  a  specific  decision.  It  is  most  important  that  the  person 
receiving  the  estimate  understand  what  he  is  getting. 

c.  What  year  dollars  are  to  be  used?  Is  there  a  requirement  for 
escalation  and/or  discounting?  If  so,  what  factors  are  to  be  used? 

The  basic  requirement  is  that  the  year  dollars  used  be  clearly  stated, 
so  that  they  can  subsequently  be  updated.  Escalation  is  a  require¬ 
ment  for  programming  documents.  The  discounting  technique  may  be 
appropriate  if  time-phased  alternatives  are  being  considered.  For 
both  escalation  and  discounting  there  are  OSD  approved  indices. 
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d.  Is  any  particular  estimating  technique  to  be  used  or  avoided? 

There  may  be  interest  in  developing  a  particular  bank  of  data  for  a  given 
system  or  type  of  system.  For  instance,  it  may  be  a  purpose  of  the 
estimate  to  develop  cost  estimating  relationships  (CER).  On  the  other 
hand,  a  well-defined  work  breakdown  structure  might  be  sought.  A  work 
breakdown  structure  is  required  of  the  project  manager  early  in  the  life 
cycle,  and  should  be  used,  if  appropriate. 

e.  What  quantities  are  to  be  used?  This  information  may  not  be 
provided  directly,  but  may  be  derived  from  other  information.  Quantities, 
for  instance,  might  have  to  be  obtained  from  a  proposed  force  structure 
and  tentative  basis  of  issue  (TBOI)  with  appropriate  logistical  factors. 

If  quantities  are  derived  in  this  manner,  care  must  be  taken  to  include 
service  school  and  other  non-TOE  requirements.  If  operating  costs  are 
included,  the  quantities  to  be  operated  must  be  identified. 

f.  Are  alternative  configurations  to  be  considered?  What  sensi¬ 
tivity  analyses  are  required?  Generally,  the  decision  maker  will  be 
interested  in  seeing  variations  to  the  basic  system  costed  out.  Identi¬ 
fying  these  variations  early  in  the  estimating  process  may  simplify  the 
data  collection  effort.  It  would  also  impact  on  the  model  selected. 

g.  How  are  risk  and  uncertainty  to  be  handled?  Particularly, 
for  those  items  early  in  the  life  cycle,  a  range  estimate,  or  a  "best 
estimate"  bounded  by  higher  and  lower  estimates  may  be  appropriate. 

h.  What  cost  displays  are  required?  The  basic  variables  involved 
are  cost  categories,  appropriation  categories,  fiscal  years,  and  work 
breakdown  structure.  In  addition,  costs  may  be  aggregated  to  meet 
particular  cost  definitions  (flyaway,  procurement,  program,  etc.). 

i.  Are  there  budgetary  constraints?  The  five  year  defense  plan 
(FYDP)  and  program  objectives  memorandum  (POM)  are  the  more  common 
sources  of  budgetary  constraints. 

j.  What  additional  assumptions  must  be  made?  What  general  factors 
are  acceptable  for  use?  In  general,  many  items  of  preliminary  guidance 
can  be  covered  by  reference  to  available  source  documents.  The  various 
commodity  commands  within  USAMG  have  factors  available  for  use  with 
their  particular  systems.  GOA  publishes  the  Army  Force  Planning  Cost 
Handbook  for  personnel  costs.  Factors  from  any  source,  particularly 
from  published  documents,  should  be  reviewed  to  insure  that  they  are 
still  current. 

The  items  of  information  mentioned  above  should  be  considered  a 
minimal  set  and  do  not  by  any  means  represent  all  the  information  the 
estimator  will  require  to  complete  the  estimate.  If  these  items  are 
not  provided  in  the  initial  guidance  from  the  tasking  organization, 
they  must  be  requested.  It  is  often  helpful  if  the  office  performing 
the  estimate  can  propose  answers  to  these  questions. 
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CHAPTER  3 


SYSTEM  DESCRIPTION  &  INFORMATION  SEARCH 


Preliminary  guidance,  although  often  inadequate,  should  at  least 
set  the  stage  for  the  cost  analyst  to  begin  to  describe  the  system 
under  study.  The  amount  and  availability  of  data  to  complete  the 
description  will  largely  depend  on  the  life  cycle  stage  of  the 
system.  Early  in  the  life  cycle,  little  other  than  conceptual  data 
may  be  available.  Late  in  the  life  cycle,  considerable  detailed  data 
can  be  found.  A  major  problem  for  the  analyst  is  to  secure  the  data 
required  to  develop  a  complete  system  description. 

Of  specific  interest,  especially  in  the  early  stages,  are  the  physical, 
performance,  technical,  and  mission  characteristics  which  the  system 
is  expected  to  achieve.  These  factors  can  provide  the  basis  with  which 
the  analyst  can  draw  comparisons  with  other  systems  for  cost  estimation 
purposes.  In  this  regard  the  Combat  Developments  Objectives  Guide 
(CDOG)  can  be  valuable  in  initially  describing  the  system  under  study. 
The  CDOG,  prepared  by  Combat  Developments  Command,  contains  all  the 
DA  approved  operational  and  organizational  objectives,  Qualitative 
Material  Requirements  (QMR),  Small  Development  Requirements  (SDR), 
and  Qualitative  Material  Development  Objectives  (QMDO).  It  further 
contains  priorities,  a  compilation  of  studies,  field  experiments,  and 
tests.  If  this  information  is  not  provided  by  the  tasking  organization, 
the  analyst  should  definitely  check  the  CDOG  data.  Even  if  the  CDOG 
does  not  provide  information  directly,  it  can  often  provide  a  lead  to 
further  information  sources. 

Later  in  the  life  cycle  many  more  documents  and  information  sources 
are  available  and  the  system  description  becomes  less  a  problem. 

Some  potential  sources  are  mentioned  later  in  conjunction  with 
information  search  for  historical  and  managerial  information. 

After  the  system  being  studied  has  been  described  as  well  as  possible, 
it  is  necessary  for  the  analyst  to  plan  the  course  of  the  study  from 
data  collection  through  report  presentation.  To  do  this  the  cost 
analyst  must  determine  as  early  as  possible  the  data  he  needs  to 
develop  his  analysis.  This  may  be  data  concerning  costs  incurred  in 
the  life  cycle  of  similar  systems,  or  it  may  be  data  from  various 
management  documents  which  affect  the  system  under  study.  The  ideal 
solution  to  these  problems  would  be  an  automated  cost  data  base  which 
would  consolidate  all  available  information  and  furnish  it  on  a  real 
time  basis  and  in  the  various  forms  required.  Unfortunately,  no  such 
system  is  yet  fully  available,  therefore,  the  analyst  must  do  most 
of  his  research  by  manual  document  search. 
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Of  considerable  benefit  then  is  a  basic  knowledge  of  some  of  the 
available  data  sources  and  the  content  of  those  sources.  The  remainder 
of  this  chapter  is  intended  to  provide  this  basic  knowledge  by  describing 
some  of  the  most  used  sources.  Those  sources  ares 

THE  FIVE  YEAR  DEFENSE  PROGRAM 

The  Five  Year  Defense  Program  or  FYDP  is  the  basic  program  document  for 
all  of  the  Department  of  Defense.  It  is  not  a  source  of  data  for 
estimating  equipment  costs,  but  a  summary  by  mission  of  the  resource 
allocation,  past  and  planned,  for  DOD.  Its  importance  to  the  weapons 
cost  analyst  springs  from  two  facts: 

1.  It  is  the  official  record  of  Secretary  of  Defense  approved 
programs. 

2.  The  structure  of  the  FYDP  influences  both  the  questions  that 
must  be  answered  in  an  analysis  and  the  records  which  DOD  maintains. 


The  FYDP  divides  up  the  activities  of  the  DOD  into  some  1400  program 
elements,  aggregated  into  ten  major  programs.  Here  you  see  the  ten  major 
programs  with  examples  of  the  program  elements: 


MAJOR  PROGRAM 

ELEMENT 

CODE 

PROGRAM  ELEMENT 

1. 

Strategic  Forces 

12518A 

Safeguard  Training 

2. 

General  Purpose  Forces 

22113A 

Infantry  Divisions 

3. 

Intelligence  and 
Communications 

31011A 

Cryptologic  Activities 

4. 

Airlift/Sealift 

43111A 

Port  Terminal  Operations 

5. 

Army  Reserves 

58112A 

Individual  Training 
(ARNG) 

6. 

Research  and  Development 

6471 4A 

Night  Vision  Development 

7. 

Central  Supply  and 
Maintenance 

71112A 

Inventory  Control  Point 
Operations 

8. 

Training,  Medical  and 

Other  General  Personnel 
Activities 

811 11A 

Recruit  Training 

9. 

Administration  and 
Associated  Activities 

91113A 

Departmental  Headquarters 

10. 

Support  of  Other  Nations 

01003A 

Thailand 
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Each  program  element  is  identified  by  a  five- digit  code  and  a  letter 
for  the  service  -  A  for  Army,  N  for  Navy,  F  for  Air  Force,  and  other 
letters  for  other  defense  agencies.  The  first  digit  of  the  program 
element  number  corresponds  to  the  ten  major  programs.  For  each 
program  element,  the  FYDP  shows  the  forces,  dollars,  and  manpower 
associated  with  it  from  the  program  element  inception  until  five 
program  years  past  the  current  year. 

Cost  are  broken  down  into  development,  investment,  and  operations, 
and  also  by  appropriation  within  these  categories.  Personnel  are 
broken  clown  into  active  officers,  active  enlisted  men,  US  direct  hire 
civilians,  and  22  other  categories  mainly  related  to  the  reserves. 

Except  for  the  R&D  program,  few  of  the  program  elements  in  the  FYDP 
are  directly  related  to  weapons — the  notable  exception  in  the  Army 
is  for  missile  systems.  As  an  example,  some  missile  systems  appear 
in  the  FYDP  whereas  tanks  do  not  because  a  missile  system  and  the 
organization  which  operates  it  can  have  an  independent  mission^ whereas 
tanks  appear  only  as  part  of  a  larger  force  such  as  an  armored 
division.  In  this  regard,  there  are  several  program  elements  directly 
related  to  the  SAFEGUARD  System,  but  even  then  it  is  not  equipment 
but  the  organization  fulfilling  a  function  which  is  identified. 

THE  ARMY  MATERIEL  PLAN 

The  Army  Materiel  Plan  (AMP)  is  used  as  a  basic  source  document  in 
developing  and  executing  the  PEMA  portion  of  the  Army  Budget.  It 
displays  equipment  requirements  determined  from  approved  unit 
authorizations  and  develops  a  yearly  procurement  schedule  to  fill 
these  requirements.  The  plan  contains  data  for  the  prior  years, 
current  year,  budget  year,  and  four  succeeding  fiscal  years.  It 
is  prepared  by  the  AMC  Subcommands  and  other  agencies  responsible 
for  PEMA  procurement.  The  AMP  matches  up  production  capability 
equipment  requirements  as  stated  in  force  unit  authorization  documents 
with  the  existing  inventory. 

The  cost  information  contained  is  only  of  a  summary  type  however,  the 
cost  analyst  can  find  the  AMP  helpful  because: 

1.  Most  equipment  items  are  covered  by  the  AMP. 

2.  The  historical  co9ts  and  cost  projections  should  be  current. 

3.  The  total  requirements  for  the  item  are  stated  based  on  current 
authorizations. 
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The  current  inventory  is  included 


5.  Tentative  procurement  plans  are  included  although  the  AMP 
itself  does  not  authorize  procurement. 

6.  Information  about  prior  and  current  contracts  is  provided. 

Information  relating  to  specific  systems  is  contained  in  six  basic  pages. 
The  first  page  summarizes  the  materiel  plan  for  the  particular  item. 

The  second  page  breaks  out  the  requirement  by  force  and  the  inventory 
(pipeline,  TOA,  etc.)  by  theater.  It  also  provides  factors  used  in  AMP 
requirements  computations.  The  third  page  gives  further  detail  on  the 
inventory  by  model  and  theater  as  well  as  a  very  brief  description  of  the 
item,  the  DA  project  number,  the  date  of  type  classification,  the  average 
useful  life,  the  age  of  the  inventory,  and  depot  repair  capability.  Page 
four  shows  procurement  and  production  information  such  as: 

Contract  type 
Delivery  date 

Initial  and  reorder  lead  time 

Economic  and  Minimum  production  quantities  with 
unit  cost 

Production  base  support 
Mobilization  requirement  and  schedule 

Page  five  contains  a  breakout  of  past  and  projected  losses  by  year  and 
loss  category.  Page  six  is  the  program  cost  breakdown. 

THE  ARMY  FORCE  PLANNING  COST  HANDBOOK 

The  Army  Force  Planning  Cost  Handbook  (AFPCH)  can  be  used  by  the  analyst 
as  an  aid  in  analyzing  variations  in  resource  requirements  resulting  from 
proposed  changes  to  approved  Army  forces  in  a  peacetime  environment.  It 
gives  a  breakout  of  costs  for  87  representative  units  including  a  list  of 
authorized  major  equipment;  however,  it  is  not  equipment  oriented.  The 
information  contained  in  the  AFPCH  will  allow  an  analyst  to  develop  a 
planning  estimate  for  the  near  future  of  costs  involved  in  changes  such 
as: 


Force  Structure 
Force  Deployments 
Activity  Rates 
Training  Requirements 
Airlift  Requirements 
Equipment  Modernization 
Logistics  Guidance 
Manpower 
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For  instance,  costs  could  be  estimated  for  adding  an  engineer 
battalion  to  a  corps  or  moving  a  division  from  CONUS  to  Europe. 

The  AFPCH  is  not  designed  for  use  in  programming  or  budgeting  but  for 
the  development  of  incremental  planning  costs  for  changes  in  the  active 
force. 

The  cost  methodology  and  sources  used  are  explained  in  the  beginning 
of  the  handbook  and  several  factors  are  given  which  may  be  of  use  to 
the  analyst  developing  operating  cost  estimates  of  new  weapon  systems. 

SELECTED  ACQUISITION  REPORTS 

The  Selected  Acquisition  Reports  (SAR)  are  summaries  of  the  current 
estimate  of  technical,  schedule,  and  cost  performance  of  selected  major 
weapon  systems  in  comparison  with  current  plans  and  the  original 
program.  The  reports  are  provided  to  Congress  and  are  the  official 
DOD  position  on  the  development  and  procurement  status  of  new  weapons 
systems.  As  such  they  receive  the  personal  attention  of  the  Secretary 
of  the  Army  and  the  Secretary  of  Defense  (or  his  Deputy)  and  thus  have 
become  the  prime  tool  by  which  top  level  management  keeps  informed  on 
new  weapons.  The  SAR's  are  updated  quarterly  by  the  project  manager 
and  reviewed  thoroughly  at  HQ  AMC,  the  DA  Staff,  the  Office  of  the 
Chief  of  Staff,  the  Army  Secretariat  and  the  Office  of  the  Secretary 
of  Defense  before  being  released  to  Congress. 

An  important  aspect  of  the  SAR  is  that  information  is  shown  concerning 
the  original  cost  estimate,  the  approved  program,  and  the  current 
estimate.  Additionally,  any  differences  are  explained.  Changes  in 
costs  must  be  specifically  explained  in  terms  of  nine  causes: 

1.  Engineering  Change.  An  alteration  in  the  physical  or 
functional  characteristics  of  a  system. 

2.  Quantity  Change.  A  change  in  quantity  to  he  procured. 

3.  Support  Change.  A  change  in  support  item  requirements. 

4.  Schedule  Change.  A  change  in  a  delivery  schedule,  completion 
date  or  intermediate  milestone  of  development  or  production. 

5.  Unpredictable  Change.  A  change  caused  by  Acts  of  God,  work 
stoppage,  Federal  or  State  law  changes,  or  other  similar  unforeseeable 
events. 

• 

6.  Economic  Change.  A  change  due  to  the  operation  of  one  or  more 
factors  of  the  economy. 


7.  Estimating  Change.  A  change  in  program  or  project  cost  due  to 
refinements  of  the  base  estimate. 

8.  Contract  Performance  Incentives.  A  net  change  in  contractual 
amount  due  to  the  contractor's  actual  performance  being  different  than  was 
predicted  by  performance  (including  delivery)  incentive  targets;  (as 
differentiated  from  cost  incentive  targets)  established  in  a  Fixed  Price 
Incentive  (FPI)  or  Cost  Plus  Incentive  Fee  Contract. 

9.  Contract  Cost  Overrun/Underrun.  A  net  change  in  contractual 
amount  over/under  that  contemplated  by  a  contract  target  price  (FPI 
contract),  estimated  cost  plus  fee  (any  type  cost  reimbursement  contract) 
or  fixed  price  rede terminable  (FPR  contract),  due  to  the  contractor's 
actual  contract  costs  being  over/under  target  or  anticipated  contract 
costs,  but  not  attributable  to  any  other  cost  growth  previously  defined. 

The  SAR  was  not  developed  as  a  source  for  cost  estimating--data  is  not 
shown  by  work  breakdown  structure  or  by  cost  element;  however,  the  analyst 
may  wish  to  use  past  SAR  data  from  similar  systems  when  developing  gross 
estimates  of  new  systems. 

The  emphasis  on  comparison  of  estimates  and  tracking  progress  is  worth 
noting.  Very  often  the  analyst  is  involved  in  more  than  a  snapshot 
picture  of  a  system--he  may  have  to  become  familiar  with  the  whole  history 
of  a  program  and  the  SAR  can  be  of  assistance. 

CONTRACTOR  REPORTS  AND  FINANCIAL  RECORDS 

Further  sources  of  data  for  the  cost  analyst  are  the  reports  which 
contractors  are  required  to  submit.  There  are  many  different  types  of 
reports  and  those  authorized  are  listed  in  the  Armed  Services  Procurement 
Regulation  (ASPR)  and  in  AR  700-51  which  contains  the  Authorized  Data 
List.  Reports  required  for  a  specific  contract  are  listed  on  DD  Form  1423, 
the  Contract  Data  Requirements  List,  and  in  Requests  For  Proposals. 

Three  of  these  reports  are  of  particular  interest  to  the  analyst.  They  are: 

Cost  Information  Reports  (CIR) 

Cost  and  Performance  Reports  (CPR) 

Production  Information  Reports  (PIR) 

*  ^ 

Cost  Information  Reports.  To  date  these  apply  only  to  aircraft,  missile, 
and  space  systems.  Their  purpose  is  to  assist  industry  and  government 
in  estimating,  analyzing,  and  tracking  the  costs  of  system  development 
and  production.  The  reporting  requirements  are  applicable  to  that 


portion  of  the  system  life  cycle  from  final  approval  for  engineering 
development  through  completion  of  production.  The  reports  are 
designed  to: 

1.  Provide  actual  costs  to  date  and  estimated  costs  at  completion 
of  contracts. 

2.  Provide  estimates  of  costs  by  fiscal  year  for  support  of  the 
Five  Year  Defense  Program. 

3.  Provide  information  in  support  of  DD  Form  633,  Department  of 
Defense  Contract  Pricing  Proposal  for  new  weapon  or  support  systems. 

4.  Provide  a  reporting  framework  in  accordance  with  the  Work 
Breakdown  Structure  (WBS). 

Five  specific  forms  may  be  required  of  the  contractor:  f 

DD  Form  1558  -  Contract  Cost  Data  Summary 

DD  Form  1558-1  -  Functional  Cost-Hour  Report 

DD  Form  1558-2  -  Progress  Curve  Report 

DD  Form  1558-3  -  Fiscal  Year  Data  Summary 

DD  Form  1558-4  -  Fiscal  Year  Functional  Cost  Hour  Report 

Cost  and  Performance  Report.  The  CPR  reflects  the  current  picture  and 
outlook  for  manhours,  funds,  and  work  completion  under  the  contract. 

It  is  used  by  the  contracting  officer  and  project  engineer  to  analyze 
contractor  progress  and  efficient  use  of  manpower  and  funds. 

The  CPR  contains  the  following: 

1.  Manhours:  Total  riianhours  expended  on  program  tasks  during  the 
month;  cumulative  total  to  date;  and  percentage  of  total  manhours  to 
date. 


2.  Funds:  Total  funds  expended,  by  task,  for  the  month; 
cumulative  total  spent  to  date;  outstanding  commitments;  and  percentage 
of  total  funds  spent  to  date. 

3.  Work  completion:  Percentage  of  work  completed,  by  tasks, 
during  the  month;  and  the  percentage  of  completion  of  total  work 
called  for  under  the  contract. 

Procurement  Information  Reports.  The  PIR  is  very  similar  to  the  CIR  - 
the  main  differences  are  that  the  PIR  is  applied  to  systems  besides 


) 
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missiles  and  aircraft,  it  covers  a  greater  number  of  systems  and  it  does 
not  necessarily  cover  the  entire  work  breakdown  structure  as  the  CIR  does. 

There  are  two  formats  in  the  PIR: 

1.  A  Progress  Curve  Report 

2.  A  Functional  Cost -Hour  Report. 

Both  are  very  similar  to  the  reports  in  the  CIR,  the  main  difference 

being  that  the  PIR  gives  a  breakdown  of  the  item  the  CIR  refers  to  as  "other" 

under  Engineering,  Tooling,  Quality  Control,  and  Manufacturing. 

Although  not  yet  fully  implemented  the  PIR  should  fill  many  of  the  gaps 
in  data  collection.  It  has  the  proper  detailed  cost  element  breakdown 
and  is  oriented  towards  WBS  packages.  It  has  the  advantage  of  brevity  over 
the  CIR  since  the  report  can  be  limited  to  selected  components  of  the 
system  in  which  there  is  particular  interest. 

These  contractor  reports  and  others  are  a  significant  source  of  data  to 
the  analyst  because  the  data  is  generally  current  and  detailed.  There 
are  some  drawbacks  though  since  reporting  requirements  vary  from  contractor 
to  contractor,  accounting  systems  and  therefore  cost  allocations  vary  from 
contractor  to  contractor,  and  the  reports  do  not  cover  all  the  information 
an  analyst  would  normally  need. 

Another  possible  source  of  data  are  the  contractors  financial  records  if 
they  are  made  available.  These  could  be  especially  helpful  to  fill  gaps 
not  covered  by  required  reports.  These  should  be  used  with  caution 
however  since  they  are  the  product  of  an  individualized  accounting  system 
and  the  assignment  of  costs  can  be  inconsistent.  The  records  would 
normally  not  include  such  items  as  costs  for  government  furnished  equip¬ 
ment  or  work  done  by  government  personnel.  Nevertheless,  if  adequate 
data  cannot  be  obtained  from  required  reports,  the  analyst  may  have  to  use 
the  financial  records  and  make  the  analysis  on  a  more  gross  level  than 
would  be  possible  with  report  data. 

WEAPONS  SYSTEM  COST  DATA  HANDBOOK  (WESCOD) 

The  Weapons  System  Cost  Data  Handbook  (WESCOD)  provides  a  summary  source 
of  cost  data  for  all  major  weapon/ support  systems.  Program  costs  are 
supported  by  data  on  schedules,  assumptions  and  quantities.  Sensitivity 
of  the  baseline  program  cost  to  key  variables  is  included  where  available 
and  the  source  of  each  cost  estimate  is  provided  for  the  user  requiring 
more  detailed  information. 
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The  cost  data  in  the  handbook  represents  the  best  estimate  at  the  date 
for  each  individual  system.  The  estimates  are  developed  using 
historical  data,  cost  estimating  relations,  current  cost  studies  and 
analyses  of  the  budgeting  cycle  and  production  process.  This  handbook 
is  updated  semi-annually. 

DEVELOPMENT  CONCEPT  PAPER  (DCP) 

The  Development  Concept  Paper  (DCP)  is  written  by  Department  of  Defense, 
Research  and  Engineering  (DDR&E)  with  the  assistance  of  the  Services. 

It  contains  the  nature  of  the  program,  the  management  issues,  current 
capabilities  and  deficiencies,  status  and  analysis  of  concept  formulation 
technology  gains,  trade-offs  and  risks,  development  alternatives, 
contracting  and  management,  and  cost,  schedule  and  performance  thresholds 
Cost  analysts  contribute  to  the  paper  and  receive  general  program 
orientation  from  it. 

AREA  COORDINATION  PAPER  (ACP) 

The  Mission  Area  Coordinating  Paper  (ACP)  is  prepared  by  Department  of 
Defense,  Research  and  Engineering  (DDR&E)  to  provide  program  guidance 
to  Military  Departments  and  Agencies,  eliminate  competing  systems, 
phase  out  obsolete  equipment  and  systems,  raise  and  resolve  major 
issues,  delineate  priorities  and  identify  major  deficiencies  and 
technological  opportunities.  This  paper  requires  data  from  cost 
analysts  and  provides  cost  analysts  with  general  orientation  for 
analytical  cost  work.  The  ACP  contains  summary  costs  from  all 
Services  in  RDT&E,  Investment  and  Operating  Costs. 

MATERIEL  NEED  (MN) 

The  Materiel  Need  (MN)  provides  development  guidance  throughout  the 
life  cycle  of  each  materiel  item.  It  is  established  through  the  joint 
efforts  of  the  combat  developer  and  the  materiel  developer  and  is 
revised  by  the  same  collaboration  throughout  the  life  cycle.  The 
Materiel  Need  (MN)  is  planned  to  replace  the  QMDO  (Qualitative  Materiel 
Development  Objective)  and  ADO  (Advanced  Development  Objective).  The 
Materiel  Need  (Engineering  Development),  MN(ED),  replaces  the  QMR 
(Qualitative  Material  Requirement,  the  SDR  (Small  Development  Require¬ 
ment  and  the  SDR-E  (Small  Development  Requirement-Expedited).  The 
Materiel  Need  (Production),  MN(P),  replaces  the  final  QMR,  SDR  and 
SDR-E. 

The  MN  (Materiel  Need)  contains  items  such  as  system  description, 
performance,  missions,  threat,  concepts  of  use,  technology,  costs, 
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time  to  develop  and  deploy  and  other  data.  The  cost  analyst  can 
contribute  cost  information  to  the  MN  and  receive  useful  data  for  cost 
analysis  from  it. 

PRIOR  STUDIES 

If  there  is  one  data  source  in  which  there  is  more  diversity  than  contrac¬ 
tor  reports  it  is  prior  studies.  A  large  number  of  studies  have  been  made 
since  the  beginning  of  systems  analysis.  Aside  from  methodological  papers, 
most  of  this  print*  1  matter  covers  case  studies  of  particular  equipment 
and  each  case  stud)  is  a  possible  source  of  data.  Most  studies  will  contain 
a  synthesis  of  data  from  several  sources  -  if  the  analyst  can  locate  the 
right  study  containing  the  right  type  of  data  in  the  right  form  he  can  save 
himself  considerable  work. 

However,  many  studies  were  not  designed  to  serve  as  data  sources  for  the 
future  analyst.  They  have  more  immediate  goals  in  mind  such  as  providing 
an  estimate  of  Research  and  Development  costs  for  a  specific  item  -  so  the 
data  is  aggregated  and  the  sources  are  given  only  in  enough  detail  to 
establish  validity.  In  general  the  freedom  to  manipulate  the  data  that  the 
analyst  who  collected  it  enjoyed  is  often  lost  to  the  reader  of  a  study. 

On  the  other  hand,  a  study  may  represent  a  compilation  of  facts  and  figures 
which  could  not  be  reproduced  from  still  existing  sources. 

There  is  no  uniformity  in  the  formats  of  most  historical  studies  so  it  Is 
often  a  time  consuming  process  to  extract  what  the  analyst  needs.  More 
standard  formats  and  documentation  requirements  should  assist  in  making 
studies  more  useful  as  historical  documents. 

While  individual  studies  can  be  an  important  source  of  data  -  the  problem 
is  often  finding  the  right  one.  Of  considerable  assistance  to  the  analyst 
is  the  Defense  Documentation  Center  which  has  many  studies  on  file  and  will 
perform  a  bibliography  search  for  the  user.  Other  experienced  analysts  are 
an  excellent  source  of  study  information  and  may  prove  to  be  most  helpful. 
Additionally,  the  Army  Materiel  Command  and  its  subcommands  along  with 
various  other  Army  activities  have  accumulated  libraries  of  past  studies. 

CAUTIONS 

In  using  data  from  different  sources  the  analyst  must  fully  understand  the 
purpose  for  which  the  data  was  originally  intended,  the  methodology  by  which 
it  was  derived  and  the  level  of  uncertainty  attached  to  it.  To  be  useful 
the  data  must  be  consistent  and  comparable  with  the  system  under  study. 

In  many  cases  the  raw  data,  as  collected,  is  neither.  Often  inconsistencies 
will  appear  in  the  form  of  definitional  differences,  yearly  price  changes, 
production  quantity  differences,  and  in  units  of  measurement. 
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In  some  contractor  reports,  for  example,  tooling  costs  for  one 
contractor  might  well  include  some  costs  that  another  contractor 
would  attribute  to  production  labor  overhead.  In  most  cases  yearly 
price  changes  would  preclude  a  direct  comparison  of  1960  costs  with  1970 
costs,  for  example,  without  adjustment.  Units  of  measurement  must  also 
be  checked  closely.  Speed  and  weight,  for  instance,  can  be  expressed 
in  a  number  of  different  terms. 

Of  possible  use  to  the  cost  analyst  in  structuring  historical  information 
to  make  it  comparable  with  the  system  under  study  and  reduce  the 
possibility  of  error  are  the  Work  Breakdown  Structure  formats  provided 
in  Military  Standard  881.  The  formats,  which  are  by  type  system 
(i.e.,  aircraft,  electronics,  missile,  etc.),  break  out  and  define 
subsystems  and  can  be  of  assistance  in  helping  insure,  for  instance, 
that  the  analyst  does  not  include  the  cost  of  guidance  equipment  and 
propulsion  equipment  of  one  system  with  guidance  equipment  alone  of 
another.  This  solves  only  a  part  of  the  problem  and  the  analyst  must 
be  continually  aware  that  to  be  usable,  data  must  be  not  only  comparable 
in  a  definitional  sense  but  also  comparable  and  consistent  in  units 
of  measurement. 
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CHAPTER  4 


TECHNIQUES  OF  COSTING 


INTRODUCTION 


This  chapter  examines  and  describes  the  techniques  by  which  cost 
estimates  are  derived.  Emphasis  is  placed  on  parametric  analysis  as 
one  of  the  techniques  with  a  detailed  discussion  of  simple  regression 
analysis  and  cost  quantity  relationships  using  specific  examples  for 
each.  Mentioned  in  less  detail  are  the  applications  of  nonlinear  and 
multiple  regression.  Additional  discussion  is  devoted  to  the  analogy, 
industrial  engineering  and  expert  opinion  approaches  to  cost 
estimating.  The  chapter  stresses  not  only  the  techniques  but  also 
the  limitations  of  those  techniques. 

Derivations  of  cost  estimates  for  weapon/ support  systems  have  taken 
place  for  many  years  by  both  the  contractor  producing  type  system 
equipment  and  the  military  services  sponsoring  the  development  of  the 
system.  These  estimates  depend  on  the  ability  of  cost  analysts  to 
establish*  relationships  between  costs  and  cost  influencing  factors. 

The  relationships  used  have  come  from  procedures  categorized  by  many 
different  name  groups!' : 


GROUP  I 


Industrial  engineering  standards 
Rates,  factors,  and  catalog  prices 
Estimating  relationships 
Specific  analogies 
Expert  opinion 

GROUP  II 

Analytical  appraisal 
Comparative  analysis 
Statistical  analysis 
Use  of  standards 

The  level  of  effort  and  knowledge  of  conditions  associated  with  these 
procedures  range  from  intuition  to  extreme  detail  depending  upon  type 
system  and  technological  innovations  proposed.  To  simplify  under¬ 
standing  and  encourage  uniformity  of  terminology  within  the  Army  Staff 
offices,  agencies  and  major  commands,  four  approaches  to  cost  estimating 
will  be  defined  and  discussed.  These  approaches  are  Industrial 
Engineering,  Analogy,  Expert  Judgment,  and  Parametric  (statistical). 

1/  "An  Introduction  to  Equipment  Cost  Estimating,  RAND,  Dec  1969 
TAD  702  424). 
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Each  procedure  has  its  place  in  the  development  of  cost  estimates, 
although  at  various  points  in  the  development  cycle  of  a  system.  The 
selection  of  the  technique  which  best  suits  the  requirements  of  the  cost 
analyst  depends  upon  several  characteristics.  Principal  among  these  is 
the  purpose  of  the  cost  estimate.  Other  important  factors  are  the  amount 
and  detail  level  of  information  available  regarding  characteristics  of  the 
system  being  costed  and  the  time  and  resources  available  to  the  organization 
developing  the  cost  estimate.  Occasionally,  a  cost  estimate  is  required  in 
a  very  short  time  period  for  a  new  type  system  on  which  there  is  no 
historical  data.  Under  these  conditions,  the  use  of  expert  Judgment  and 
the  analogical  technique  is  the  only  method  that  can  meet  the  established 
constraints.  If  applicable  CER's  were  developed  in  advance  and  readily 
available,  the  use  of  a  parametric  approach  might  also  be  able  to  meet  the 
highly  constrained  conditions  of  time  and  very  limited  information.  Should 
detailed  information  on  the  system  be  available,  the  system  not  advancing 
the  state-of-the-art,  and  one  is  seeking  to  establish  a  basis  for  maintaining 
control  over  system  production,  the  engineering  estimate  is  probably  the 
most  desired  approach  for  developing  a  cost  estimate.  .  Finally,  during  the 
Concept  Formulation  phase  of  system  development  when  conditions  are  such 
that  only  physical  and  performance  system  characteristics  are  established, 
the  cost  estimate  can  be  developed  over  a  short  period  of  time,  and  limited 
manpower  is  available  to  develop  the  estimate,  use  of  the  parametric 
technique  is  the  preferred  approach. 

In  addition  to  these  topics,  the  use  of  expert  Judgment  is  discussed  within 
the  context  of  using  it  in  the  estimating  process.  In  using  the  data 
generated  through  expert  judgment,  the  analyst  needs  to  keep  in  mind  £he 
state  of  technology  applicable  to  the  system  being  developed,  whether  or 
not  other  data  are  actually  available,  and  whether  the  advantages  of  expert 
Judgment  outweigh  the  disadvantages  (e.g.,  extreme  subjectivity,  non- 
mathematical,  etc.)  for  the  situation  at  hand. 

PARAMETRIC  COST  ESTIMATING 

As  already  mentioned,  Parametric  estimating  is  the  preferred  procedure  to 
be  used  in  deriving  a  cost  estimate  at  the  concept  formulation  stage  of 
materiel  development.  At  this  stage,  system  costs  can  only  be  based  on 
ascribed  physical  and  performance  characteristics  and  their  relationship 
to  highly  aggregated  component  costs.  In  other  words,  a  functional 
'•elationship  must  be  set  up  between  total  system  cost  (estimate)  an)  the 
various  system  characteristics  or  parameters  (c=f (ascribed  characteristics)). 
In  the  present  context  the  term  parameter  means  a  cost  related  explanatory 
attribute  which  may  assume  various  values  during  a  particular  calculation. 
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For  cost  analysis  purposes,  a  parameter  is  a  definable  characteristic 
of  a  device,  item,  or  system.  The  relationships^  under  discussion 
may  be  arrived  at  through  two  processes,  either  empirical  or  statistical. 
Many  cost/parametric  relations  may  be  observed  from  historical  data. 

In  using  statistics  to  develop  cost/parameter  relations  from  cost 
histories  of  prior  programs,  two  things  must  be  kept  in  mind: U  (1)  the 
uncertainty  inherent  in  any  application  of  statistics,  and  (2)  whether 
the  resulting  relationship  is  reasonable  and  structurally  sound.  The 
first  is  unavoidable,  however,  the  influence  of  the  second  can  be 
diminished  through  careful  checks  of  the  derived  estimating  relation¬ 
ships  through  inspection,  simple  test  data  plots,  or  by  more  complicated 
techniques  which  involve  looking  at  each  parameter  over  a  range  of 
possible  values.  For  example,  one  would  not  expect,  in  arriving  at  a 
relationship  between  missile  cost  and  the  parameters  of  weight  and 
range,  that  the  cost  would  be  inversely  associated  with  range.  Such 
a  situation  would  have  to  be  investigated  and  corrected  before  attempting 
to  derive  an  actual  predictive  estimate.  The  Office  of  the  Secretary 
of  Defense  has  recognized  that  although  the  requirements  of  the 
programming  system  makes  point  estimates  of  cost  mandatory  during  the 
concept  formulation  phase  of  system  development,  a  range  of  values  would 
give  a  more  realistic  picture  of  a  new  system's  resource  requirements. 
Thus,  the  point  estimate  given  in  Development  Concept  Papers  (DCP's) 
for  the  concept  formulation  stage  and  in  SAjR's  during  later  stages  of 
system  development  should  be  developed  parametrically  and  footnoted 
with  an  interval  of  values.  Three  possible  interval  forms  are 
available:  (1)  the  confidence  interval,  (2)  the  interval  based  upon 
the  standard  error  of  the  estimate,  and  (3)  the  prediction  interval.— 

The  last,  the  prediction  interval,  is  recommended  for  use  during  the 
early  portion  of  the  concept  formulation  phase  since  uncertainty  of 
data  is  highest  during  this  period.  The  prediction  interval  is  a 
statement  about  the  true  value  of  system  cost  (C)  and  essentially 
combines  the  features  of  the  confidence  and  standard  error  intervals. 


COST/PARAMETER  RELATIONSHIPS 


The  following  portion  of  this  section  will  be  devoted  to  approaches 
for  establishing  cost/parameter  relationships.  The  initial  discussion 

1/  Once  this  relationship  has  been  developed,  the  process  of  develop¬ 
ment  should  be  documented  to  provide  an  audit  trail  of  the  cost  estimates 
obtained  from  its  use.  The  steps  on  documentation  contained  in  the 
RAND  publication,  "An  Introduction  to  Equipment  Cost  Estimating"  page 
73  provide  an  excellent  reference  in  this  matter. 

t 

2/  "An  Introduction  to  Equipment  Cost  Estimating,  RAND,  Dec  1969 
TAD  702424). 

3/  This  topic  is  discussed  in  the  document,  "The  Use  of  Classical 
Statistics  in  Deriving  and  Evaluating  CER's"  published  by  OSD(SA),  Oct 
1966  (AD  659323),  in  greater  detail  in  Lindgren,  B.W.,  Statistical 
Theory,  Macmillan,  New  York,  1962,  page  371  and  also  in  the  RAND 
Memorandum,  "An  Introduction  to  Equipment  Cost  Estimating,  Dec  1969 
(AD  702424). 
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is  limited  to  a  simple  two  variable  linear  regression  model  and  will  later 
briefly  cover  extensions  into  multiple  and  non-linear  regression.  Model 
derivation  will  be  briefly  discussed  since  an  understanding  of  theory  is 
certainly  good  background.  However,  the  most  important  thing  to  be  gained 
by  practitioners  is  an  appreciation  of  the  strengths  and  limitations  of 
regression  analysis  and  the  pitfalls  to  be  avoided  in  its  application. 

Thus,  this  discussion  will  try  to  stress  those  aspects.  Anyone  can  simply 
plug  raw  data  into  equations  and  come  up  with  an  answer  -  good  cost 
estimation  extends  far  beyond  that  into  meaningful  application  and  analysis. 

By  way  of  background  it  would  be  well  to  clarify  some  definitions  - 
specifically  the  differences  between  "regression"  and  "correlation"  since 
the  two  are  frequently  used  interchangeably.  Regression,  as  it  has  become 
known,  is  concerned  with  determining  the  equation  of  the  line,  plane,  or 
curve  relating  to  two  or  more  variables,  and  the  variance  associated  with 
any  predictions  to  be  made  from  the  model.  Correlation  has  become  associated 
primarily  with  a  correlation  coefficient  which  measures  the  closeness  with 
which  variables  are  related. 

With  all  this  in  mind  consider  the  problem  at  hand  -  cost  estimating. 

It  will  be  best  to  do  that  by  tracking  through  with  a  hypothetical  example. 

As  cost  analysts,  we  are  asked  to  estimate  the  initial  tooling  costs  for 
a  new  weapons  system  having  these  specific  design  characteristics: 

Maximum  Speed  (Hwy)  65  MPH 

Load  Capacity  7.0  Ton-Miles/Gal  of  fuel 

Weapon  System  Weight  10,000  Lbs 

Suppose  the  following  hypothetical  tooling  costs  are  our  historical  data 
base,  along  with  physical  and  performance  characteristics  for  14  other 
weapon  systems  already  past  the  initial  tooling  stage  as  shown  in  Figure  1. 


INITIAL  TOOLING  COST  AND  VARIOUS  WEAPON  SYSTEM  CHARACTERISTIC  DATA 


Weapon 

System 

Type 

Initial 
Tooling  Cost 
(Millions  $) 

Weapon  System 
Weight 

(Hundreds  of  Lbs) 

Maximum 
Speed  (mph) 

Otoy) 

Load  Capacity 
(Ton-miles/Gal) 

L-l 

28 

27 

43 

1.8 

L-2 

35 

28 

48 

2.0 

L-3 

40 

29 

50 

1.5 

M-l 

60 

35 

65 

2.0 

M-2 

50 

32 

70 

5.0 

M-3 

55 

40 

100 

2.6 

M-4 

90 

45 

110 

4.5 

H-l 

70 

60 

43 

17.0 

H-2 

285 

135 

45 

29.0 

H-3 

130 

70 

100 

21.0 

H-4 

105 

90 

43 

9.0 

H-5 

80 

70 

23 

9.0 

H-6 

40 

40 

40 

7.0 

H-7 

185 

110 

45 

6.8 

Source:  Hypothetical  Data 

PROBLEM:  Conduct  a  simple  linear  regression  analysis  of  initial  tooling 
cost  vs  weapon  system  weight  using  the  data  given  above. 
Specifically,  develop  the  following*: 

(1)  The  estimating  or  regression  equation 

(2)  The  standard  error  of  estimate 

(3)  The  coefficient  of  correlation 

*  Application  of  statistical  procedures  found  in  any  statistics  book 
will  allow  the  reader  to  also  determine  the  coefficient  of  variation 
and  a  confidence  interval  around  the  regression  line  derived  from  the 
above  data. 


-  Figure  4-1 


What  is  wanted  is  to  derive  an  equation  relating  tooling  costs  to  one 
weapon  system  characteristic  and  use  that  to  estimate  cost  of  new 
weapon  systems. 

SCATTER  DIAGRAMS 

The  first  step  is  to  simply  plot  historical  data  on  a  scatter  diagram. 
Wanting  to  determine  cost,  that  becomes  our  dependent  variable,  and  will 
be  represented  on  the  Y  axis.  The  related  performance  characteristic 
will  be  the  independent,  or  known  variable,  and  will  be  represented  on 
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X  axis.  The  purpose  of  the  scatter  diagram  is  to  insure,  before  making  ^ 

the  computations,  that  there  is,  in  fact,  a  linear  relationship  between 
the  variables.  Having  data  on  three  characteristics,  the  objective  is  to 
pick  the  variable  with  the  closest  apparent  linear  relationship  to  cost. 

The  historical  data  plotted  against  cost  is  shown  in  Figure  2. 

Apparently  the  best  relationship  is  between  weapon  system  weight  and  cost; 
so  that  will  be  used  for  the  example  analysis.  The  purpose  now  is  to 
determine  the  best  linear  relationship  between  historical  tooling  cost  and 
weapon  system  weight,  in  order  to  predict  the  tooling  cost  of  the  new 
weapon  system.  Before  doing  that  though,  it  would  be  well  to  pause  at 
this  point  and  observe  some  cautions. 

While  having  glossed  over  the  source  of  data,  one  should  be  aware  from 
the  previous  discussion  on  information  sources  that  locating  good,  reliable, 
historical  data  is  often  a  major  problem  in  itself.  Let  it  be  said  here 
that  results  can  be  no  more  reliable  than  the  input  data,  and  care  should 
be  used  in  selecting  that  data.  Care  must  also  be  taken  to  insure  that  the 
variables  used  are,  in  fact,  related.  As  an  example,  someone  once  compared 
beer  consumption  with  church  attendance  and  found  that  there  was  a  strong 
positive  correlation  between  the  two.  His  conclusion  was  that  lowering 
the  price  of  beer  would  increase  church  attendance.  The  point  is  to  insure 
from  the  onset  that  the  variables  are  related  to  each  other  so  that  you  do 
not  end  up  with  an  interesting  but  useless  study. 
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Figure  4-2 


Figure  4-2  (Continued) 
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Another  caution  to  be  considered  is  in  the  consistency  of  the  sample 
data.  Suppose  our  scatter  gram  looked  instead  like  this: 


Weight 


While  it  would  otherwise  have  an  apparently  good  linear  relationship,  this 
single  point  with  a  wide  variance  from  the  other  points  will  considerably 
skew  the  final  answer  to  the  high  side.  Yet,  to  simply  exclude  this  point 
from  the  analysis,  the  result  may  end  up  with  an  estimate  that  is  too  low. 

There  is  no  hard  and  fast  rule  in  cases  such  as  this,  except  that  the  point 
must  be  investigated.  Upon  investigation  it  might  be  found  that  the  weapon 
system  (or  some  component)  was  plagued  with  unusual  design  or  tooling 
problems  which  are  not  likely  to  recur.  The  analyst  might  also  find  that 
what  was  labeled  as  "tooling  costs"  really  included  other  costs  as  well. 

In  such  cases  it  may  be  justifiable  to  not  consider  that  point  in  the  analysis. 
Whatever  the  conclusions,  the  final  decision  to  include  or  exclude  that  point 
will  be  a  subjective  one,  and  oriented  towards  insuring  that  the  data  used 
is  the  best  representation  of  the  situation  encompassing  the  new  system. 

To  be  useful,  the  data  must  also  be  consistent  and  comparable.  In  most 
cases  the  raw  data,  as  collected,  is  neither.  Therefore,  before  estimates 
can  be  derived,  an  adjustment  must  be  made  for  yearly  price  changes,  defi¬ 
nitional  differences,  and  units  of  measurement.  As  an  example,  weight  can  be 
in  terms  of  gross  weight  or  empty  weight  with  a  big  difference  between  the 
two.  As  another  example,  labor  costs  rose  30%  between  1953  and  1960. 

Therefore,  these  differences  must  be  adjusted  so  that  all  elements  in  the 
sample  are  defined  in  common  units  and  in  dollars  adjusted  to  a  specific 
time  period. 


All  these  considerations  take  much  time  and  effort  and  the  example  has  not 
even  gotten  to  the  analysis  yet.  However,  it  is  absolutely  necessary  that 
these  cautions  be  observed  carefully  because  all  future  conclusions  in  the 
analysis  are  based  entirely  on  the  sample  data.  Any  errors,  inconsistencies, 
or  bias  in  the  sample  will  introdue  unnecessary  error  into  what  is  already 
an  imprecise  analysis. 

THE  REGRESSION  MODEL 

Being  satisfied  that  weapon  system  weight  has  the  best  relationship  to 
cost  and  that  the  necessary  cautions  have  been  observed,  the  analysis  may 
proceed.  The  equation  of  the  regression  model  is 

y  =  a  +  bX,  (1) 

where  y  is  the  dependent  variable  and  x  is  the  explanatory  variable.  The 
symbols  a  and  b  are  the  constant  and  coefficient,  respectively,  of  the 
equation  estimated  from  the  data.  Here  y  could  represent  the  cost  of 
initial  tooling  and  x  could  represent  the  weapon  system  weight.  If  it  is 
assumed  that  b  is  greater  than  zero,  the  model  indicates  that  heavier 
equipment  will  cost  more  than  lighter  equipment.  When  the  values  of  a 
and  b  are  known,  it  is  possible  to  compute  y  (cost)  for  any  given  value 
of  x  (weight). 
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Least-Squares  Estimating 

Given  Eq.  (1),  the  basic  problem  in  the  first  phase  of  the  regression 
analysis  is  to  derive  estimates  of  the  parameters  a  and  b.  £his 
could  be  done  simply  by  visually  estimating  the  relating  line,  than 
determining  the  slope  (b)  and  y  intercept  (a).  The  disadvantage 
though  is  that  all  people  see  things  differently  and  very  likely 
no  two  line  estimates  would  agree.  Therefore,  it  would  be  better  to 
have  a  more  precise  and  standard  method  of  estimation.  The 
standard  procedure  is  the  method  of  least -squares.  The  values  of 
a  and  b  are  determined  by  the  requirement  that  the  sum  of  the  squares 
of  the  deviations  of  the  sample  observations  from  the  estimated  line 
will  be  at  a  minimum.  Symbolically,  this  minimum  is  expressed  as 


i=l 


Where  y^  is  the  ith  observation  and  y^  is  the  value  of  y$  estimated 
from  the  equation 

y^_  =  a  +  b  xi  (3) 

The  carets  over  a  and  ^  indicate  that  a  and  ^  are  least-squares 
estimates  of  the  true  but  unknown  values  of  a  and  b.  Thus,  y^  is 
the  least  squares  estimate  of  yj  and  the  form  (yi-yj)  indicates  the 
difference  between  each  observed  y^  and  between  each  corresponding 
estimated  value  y*.  This  is  illustrated  in  Fig  3,  which  shows  the 
actual  (y)  and  estimated  (y)  value  of  the  dependent  variable  that 
corresponds  to  a  specific  value  of  the  explanatory  variable  x.  The 
line  shown  in  the  figure  is  the  line  that  represents  Eq.  (3).  All  of 
the  estimated  values  of  y*  fall  on  this  line.  The  vertical  distance 
from  point  A  to  point  B  is  the  difference  between  the  actual  value  (y) 
and  the  estimated  value  (y).  The  summation  of  all  such  differences 
that  are  squared  (as  illustrated  in  Eq.  (2))  is  the  quantity  to  be 
minimized  in  estimating  the  line. 

The  minimum  value  for  this  sum  is  satisfied  by  substituting  Eq.  (3) 
in  Eq,  (2),  taking  the  partial  derivatives  of  Eq.  (2)  with  respect  to 
a  and  b,  and  setting  the  results  equal  to  zero. 

This  process  yields  two  equations  that  are  called  normal  equations  and 
that  can  be  solved  for  a  and  b: 


=  na  +  b£ 

(4) 

2T  x^  yj^  =  a2!xi  +  bXx^  , 

(5) 
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Deviation  of  actual  value  from  estimated  value  and  sample  mean 


Figure  4-3 


where  y ±  = 

xi  - 
n  — 


cost  of  initial  tooling  in  millions 
weight  of  weapon  system  in  hundreds 
Number  of  items  in  the  sample, 
summation  (e.g.,  y  =  the  sum  of  all 


of  dollars, 
of  pounds. 


y 's). 


Assuming  that  the  hypothetical  data  used  in  this  example  were  reliable, 
we  now  have  everything  necessary  for  determining  the  equation  of  our 
model.  Figure  4  contains  the  numerical  values  and  totals  required  to 
solve  the  normal  equations. 


The  costs  are  expressed  in  millions  of  .dollars  and  weights  in  hundreds 
of  pounds.  Substituting  those  values  into  the  normal  equations,  the 
following  expressions  are  obtained  for  the  sample  data  points: 

1253  =  14a  +  811b 

101621  =  811a  +  61653ft 


Solved  simultaneously  these  give: 

a  =  -25.140 
ft  -  1.979 

So  the  model  is: 

y  =  -25.140  +  1.979x 

The  line  representing  this  equation  is  shown  here  with  the  actual 
observations  plotted  on  figure  5. 

One  now  has  the  regression  function  -  obtained  by  fitting  a  line  to 
the  scatter  diagram.  That  is,  point  prediction  of  the  expected 
tooling  cost  for  the  new  weapon  system  could  now  be  made  but  without 
being  able  to  attach  any  degree  of  confidence  in  the  estimate.  The 
regression  line  in  itself  is  insufficient  since  no  sampling  errors 
have  been  taken  into  account. 
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Figure  4-4 
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MEASURING  ERRORS 


The  scatter  of  observations  about  it  relates  inversely  to  the  usefulness 
of  the  line  as  a  tool  for  estimating  the  values  of  y  from  the  values  of 

x.  The  greater  the  dispersion  of  observed  values  of  y  about  the  line, 
the  less  accurate  the  estimates  that  are  based  on  the  line  are  likely  to 

be.  Looking  at  it  another  way,  it  can  be  said  that  the  regression  line 
is  an  estimate  with  error.  Fortunately,  this  error  can  be  measured  and 
displayed  in  difference  forms.  This  then  is  one  of  the  advantages  of  the 
statistical  approach  over  the  analogy  or  engineering  approaches. 

One  way  to  display  the  error  is  to  do  it  in  terms  of  the  regression 
line  itself.  This  is  called  the  standard  error  of  the  estimate  and  it 
measures  the  average  error  of  the  regression  line  in  providing  estimates 
of  y  (cost)  from  given  values  of  x  (weight). 


The  standard  error  of  estimate  is  defined  as  the  square  root  of  the 
unexplained  variance  of  the  y's:  i- ■  -  - A . - 

l(y±  -  V  2 

SE  _ 

Y  n  -  2  (6) 


where  the  term  "n-2"  is  defined  as  the  "degrees  of  freedom." 

For  the  equation  y  =  -25.140  +  1.979  x,  the  standard  error  of  estimate 
is  25.01.  This  value  has  been  plotted  above  and  below  the  regression 
line  in  Figure  5. 

This  measure  of  reliability  has  been  considered  in  the  context  of  the 
regression  equation  in  relation  to  the  sample  observations.  This  is 
usually  not  the  context  that  is  of  greatest  Interest.  Rather  than  being 
concerned  with  how  well  the  regression  equation  describes  the  sample 
observations  per  se,  the  analyst  is  most  often  interested  in  using  the 
estimating  equation  to  predict  values  of  y  in  the  "population"  or 
"universe"  that  the  sample  supposedly  represents.  While  having  described 
the  sample  well  it  cannot  yet  be  extended  to  the  new  system  which  is  not 
part  of  the  sample.  In  the  context  of  prediction,  the  standard  error  of 
estimate  does  not  furnish  a  good  measure  of  uncertainty  or  reliability 
of  the  estimating  (regression)  equation.  In  a  formal  sense,  what  would 
be  desired  is  somewhat  as  follows.  For  a  given  value  of  the  explanatory 
variable,  say  Xj.,  the  estimating  equation  is  used  to  obtain  a  predicted 
value  of  the  dependent  variable: 

$  =  a  +  b  xt  (7) 

It  would  now  be  necessary  to  put  a  boundary  around  y  --  say  y  +  A  --  such 
that  there  is  a  certain  level  of  confidence  that  the  established  interval 


does  indeed  "bracket"  the  "true"  value  of  Y  in  the  population.  The 
subject  of  "prediction  intervals"  discussed  on  page  4-3  and  Footnote  3 
of  page  4-3  addresses  this  problem.  Symbolically  the  prediction  interval 
for  a  simple  linear  regression  is  represented  as  follows: 

yt  refa)/lii » 

J  *  .Tfa-S)1 

Where:  SE  =  standard  error  as  previously  defined 

t  =  value  of  the  t  statistic  at  the  confidence  level  desired 
n  =  size  of  sample 
x  =  mean  of  the  x^s  in  the  sample 

x  =  the  specified  value  of  the  explanatory  variable  used  as 
a  basis  for  obtaining  y. 

Of  significance  then  is  the  fact  that,  to  be  objective,  predictions 
based  on  regression  analysis  should  include  a  confidence  Interval.  That 
interval  is  highly  sensitive  to  the  level  of  confidence  specified,  the 
size  of  the  sample,  and  the  distance  of  X^  from  X. 

Another  way  of  displaying  the  error  involved  in  regression  analysis  is 
in  terms  of  the  regression  coefficient  itself.  Previously,  it  was 
determined  from  the  regression  equation  that  b  =  1.9789.  This,  of  course, 
is  a  point  estimate  so,  to  display  the  error  in  these  terms,  it  is 
requested  to  establish  a  confidence  Interval  around  b.  One  might  find 
that  a  957.  confidence  interval  is  from  1.5300  to  2.4278.  This  means 
of  displaying  regression  analysis  errors  is  not  normally  used  in  cost 
analysis  studies  however. 

Up  to  this  point,  the  discussion  has  been  confined  largely  to 
statistical  regression  analyses  --  developing  an  estimating  (regression) 
equation  and  various  measures  of  uncertainty  pertaining  to  that  equation. 
From  an  estimating  point  of  view,  this  indeed  is  the  most  important  part 
of  the  analysis. 

CORRELATION  ANALYSIS 

There  is,  however,  another  form  of  statistical  analysis  called 
correlation  analysis.  Correlation  analysis  is  concerned  with  developing 
an  abstract  measure  of  the  degree  of  association  between  the  dependent 
variable  and  the  explanatory  variable  (or  variables).  In  simple  linear 
regression,  the  most  commonly  used  measure  of  degree  of  association 
is  the  correlation  coefficient,  denoted  by  r.  The  statistic  r  is 
constructed  in  such  a  way  that  it  is  bounded  by  the  interval 
-l£r  5-+1.  The  sign  indicates  whether  the  slope  of  the  regression 
line  is  positive  or  negative  —  i.e.,  whether  the  regression  co¬ 
efficient  b  is  positive  or  negative.  At  the  boundaries  of  the  interval 
for  r,  there  are  the  cases  of  perfect  correlation:  r  =  +1  (perfect 
positive  correlation,  see  Figure  6),  r  =  -1  (perfect  negative 
correlation,  see  Figure  6).  In  these  instances  all  of  the  sample 
points  would  lie  exactly  on  the  regression  line.  When  there  is  no 
correlation  between  the  variables  whatsoever,  r  =  0,  see  Figure  6. 
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While  correlation  is  a  somewhat  different  type  of  analysis  from  that 
discussed  previously,  it  is  nevertheless  related  in  a  definite  way  to 
regression  analysis.  In  order  to  see  this,  recall  the  concepts  of 
total  variance,  explained  variance,  and  unexplained  variance  referred 
to  earlier  in  the  discussion  of  the  standard  error  of  estimate. 

Total  variance  pertains  to  the  deviations  of  the  sample  Y  '  s  from  their 
mean,  and  is  measured  by: 

71  <Yi  .  y)2 

N  (8) 

—  A 

Explained  variance  refers  to  the  deviations  from  Y  of  the  computed  Y^ 
values  (calculated  from  the  regression  equation)  corresponding  to 
the  values  of  X  in  the  sample,!.'  and  is  measured  by: 

XA  -  «2 

N  (9) 

Unexplained  variance  is  derived  from  the  deviations  of  the  sample  Y^ 
values  from  the  computed  values  of  and  is  measured  by: 

'Xo'i  -  fy2  y 

N  ,(10) 

Intuitively  one  would  think  that  the  standard  error  of  estimate  might 
somehow  be  derived  from  the  unexplained  variance.  From  previous 
discussion,  it  was  noted  that  this  is  indeed  the  case.  The  standard 
error  of  estimate  (unadjusted)  is  the  square  root  of  the  unexplained 
variance. 

Similarly,  one  would  intuitively  think  that  the  correlation  coefficient 
(r)  might  somehow  be  derived  from  the  unexplained  variance.  The 
correlation  coefficient  is  in  fact  related  to  the  explained  variance. 


1/  That  is,  for  each  value  of  X  in  the  sample,  a  corresponding  value 
of  the  dependent  variable  (Y)  is  computed  from  the  regression  equation 
Y  =  a  +  bX. 

2/  A  graphic  portrayal  of  total,  explained  and  unexplained  variance 
is  contained  in  Croxton  and  Cowden,  ££.  clt. ,  pp.  662-63. 
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It  is  defined  as  the  square  root  of  the  proportion  of  total  variance 
that  is  represented  by  the  explained  variance.!/  That  is 


I(*i  *  V 


X(Yt  -  Y.V 


I(Yt  -  Y)2 


2I(Y1  -  Y)2 


Now  the  interrelationship  among  r,  S,  and  the  regression  equation  becomes 
more  clear.  The  regression  equation  is  used  to  determine  the  computed 
Y's,  which  are  inputs  to  the  calculation  of  both  r  and  SE.  Also,  since 
r^  is  defined  as  a  proportion  of  total  variance,  r  and  SE  in  a  sense  have 
an  inverse  relationship  to  one  another. 

Just  as  the  confidence  interval  had  to  be  adjusted  for  sample  size  -- 
particularly  so  in  the  case  of  small  samples  --  the  correlation  coefficient 
should  also  be  corrected.  In  the  case  of  simple  linear  correlation  the 
value  of  r  corrected  for  sample  size  is  as  follows: 


-/  -  y>2 

,  /  Zch  -  W 


(n-l)-l 


As  is  obvious  from  this  equation,  the  effect  of  the  correction  dampens 
out  as  n  becomes  larger.  For  very  small  samples,  as  in  our  illustrative 
example,  the  correction  should  most  certainly  be  made. 

Sufficient  information  is  now  available  to  compute  the  correlation 
coefficient.  Adjusted  for  sample  size  the  data  is  as  follows: 


57468 

64966 


(14-1)  -  1 


•8846(13)-1 

12 


(14-2; 


11.50-1 

12 


=  .8749 


.8749 


=  .9354 


is  sometimes  referred  to  as  the  coefficient  of  determination. 
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The  fact  that  r  =  ,94  seems  rather  impressive.  This  represents  a 
rather  high  correlation.  But  it  is  easy  to  be  misled  by  high 
correlation  coefficients.  So  much  so,  that  in  recent  years  there  has 
been  a  trend  away  from  the  former  emphasis  on  correlation  analysis 
per  se  to  regression  analysis  which  stresses  the  derivation  of 
structurally  sound  estimating  relationships  and  of  measures  of  the 
confidence  that  the  user  might  have  in  the  estimating  equations.  One 
reason  is  that  the  correlation  coefficient  is  somewhat  misleading. 

For  example,  a  correlation  coefficient  of  .70  may  seem  to  be  acceptable 
but  it  really  implies  that  only  49%  of  the  total  variance  is  explained 
by  the  regression  equation.  Put  in  these  terms,  it  may  not  be 
acceptable. 

OTHER  REGRESSION  TECHNIQUES 

This  completes  the  example  of  a  simple,  2  variable  linear  regression 
model.  As  one  might  expect,  it  is  often  true  that  a  dependent 
variable  is  more  accurately  a  function  of  two  or  more  variables, 
rather  than  simply  one.  It  may  be  possible  then,  to  derive  a  more 
accurate  estimate  of  the  cost  of  a  new  system  by  considering  not  one, 
but  several  variables  which  will  affect  the  new  system  by  considering 
not  one,  but  several  variables  which  will  affect  the  cost.  In  this 
case,  a  multiple  regression  model  would  be  appropriate. 

The  approach  taken  is  similar  to  that  used  in  the  simple  regression 
model,  except  that  the  number  of  normal  equations  will  be  increased. 

If  one  understands  the  simple  regression  technique,  multiple  regression 
is  easily  understood.  Because  of  this,  and  since  multiple  regression 
is  adequately  covered  in  most  elementary  statistics  texts,  it  will 
not  be  formulated  here. 

Multiple  regression  also  provides  an  excellent  vehicle  for  sensitivity 
analysis  -  an  area  of  increasing  interest  to  cost  analysts.  As  an 
example,  a  determination  could  have  been  made  of  the  net  effect  on 
cost  of  changes  in  all  three  characteristics:  weight,  speed,  and 
Load  Capacity.  Multiple  correlation  could  have  provided  ''net 
regression  coefficients"  which  indicate  the  change  in  cost  per  unit 
change  in  each  of  the  performance  characteristics,  holding  the  other 
two  independent  variables  constant.  It  would  then  be  known  to  which 
of  the  performance  characteristics  our  cost  estimate  is  most  sensitive. 
Further  analysis  might  then  be  in  order  to  determine  the  probability 
of  changes  to  the  sensitive  factor.  That  is,  if  cost  is  highly 
sensitive  to  weight,  and  if  there  is  a  good  probability  that  the  weight 
of  the  new  weapon  system  may  increase,  we  would  be  remiss  in  simply 
assuming  a  fixed  weight. 
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Next,  the  analyst  should  be  aware  that  many  cost  relationships  are  not 
linear,  and  that  forcing  non-linear  data  into  a  straight  line  simply 
Introduces  greater  error  into  estimations.  Curvilinear  methods  of 
regression  should  be  used  if:  (1)  logic  of  the  association  between  two 
variables  supports  a  curve  rather  than  a  line,  and  (2)  the  standard  error 
of  the  estimates  is  less  than  for  a  straight  line.  The  initial  scatter 
diagram  should  provide  the  first  clue  that  a  curve  rather  than  a  line  is 
a  better  model  for  use.  Curvilinear  regression  techniques  are  covered 
in  most  elementary  statistical  texts  and  are  similar  to  simple  linear 
regression.  The  simplest  solution  technique  is  to  convert  inputs  to 
logarithims  and  thereby  change  the  problem  into  one  which  can  be  solved 
by  the  linear ^technique  just  gone  through.  Taking  the  anti-log  of  the 
final  answer  will  provide  a  solution  in  non-linear  terms. 

Regression  analysis  can  be  laborious  and  time  consuming  if  done  by  hand, 
as  done  here.  Many  of  the  computers  available  today  are  already  programmed 
for  simple  and  multiple  regression  problems  and  the  practicing  analyst 
will  undoubtedly  prefer  their  use.  The  use  of  computers  is  preferable 
since,  especially  in  problems  of  any  complexity,  errors  are  reduced  and 
time  spent  in  computations  arereduced.  Some  cautions  are  necessary  though 
and  those  will  be  covered  later. 

REGRESSION  SUMMARY 

In  summary,  regression  analysis  is  a  powerful  tool  for  establishing  cost 
estimating  relationships,  if  used  correctly.  Regression  analysis  provides 
three  vital  pieces  of  information  which  are  of  increasing  interest  to 
cost  analysts  at  all  levels.  It  provides  a  mathematical  basis  for  making 
an  estimation,  a  means  of  assessing  the  degree  of  uncertainty  in  the 
estimate,  and  it  provides  an  excellent  sensitivity  analysis  vehicle. 

PITFALLS 

Some  factors  to  consider  and  pitfalls  to  be  avoided  when  developing  cost 
estimating  relationships,  or  when  reviewing  someone  else's  CER's  are  as 
follows: 

1.  Be  aware  of  the  source  of  the  estimate,  and  the  purpose  for 
which  it  is  intended.  Regardless  of  the  integrity  of  the  individual 
analyst,  it  should  be  expected  that  some  personal  or  organizational 
bias  may  creep  in.  Contractors  naturally  want  to  sell  their  products 
or  services,  and  their  interests  may  be  served  by  low  estimates. 
Occasionally,  some  activities  may  be  served  by  a  high  estimate.  In 
this  regard  you  could  have  an  advantage  if  you  have  a  broad  range  of 
historical  cost  data  from  several  sources,  while  the  same  data  may  not 
be  available  to  private  contractors.  Application  of  statistical  analysis 
or  simple  analogy  can  give  you  an  excellent  means  of  checking  estimates 
provided  by  other  sources. 


4-20 


2.  A  simple  check  of  the  equations  used  In  cost  estimating 
relationships,  along  with  common  sense,  will  often  Indicate  whether 

or  not  the  relationship  is  a  reasonable  one.  The  pitfall  to  be  avoided 
is  that  an  equation  may  adequately  describe  one  system  but  not  be 
predictive  of  another.  As  an  example,  the  following  was  developed  by 
students  at  the  Air  Force  Institute  of  Technology  to  describe  the  cost 
model  for  small  missiles: 

C  =  8347.5  +  150. 6W  -  1149. 1R 
where  C  =  cost  of  airframe  plus  guidance  and  control 
W  =  weight 
R  =  range 

The  use  of  this  relationship  to  estimate  the  cost  of  a  light,  long 
range  missile  would  lead  to  the  conclusion. that  its  cost  would  be  less 
than  a  heavier,  short  range  missile.  This  seems  unlikely,  so  further 
examination  of  the  model  would  be  in  order. 

3.  Consistency  of  data  used  is  essential.  When  combining  data 
for  a  regression  sample,  for  instance,  it  is  usually  necessary  to 
adjust  dollar  figures  into  constant  dollars.  Because  labor  and 
materials  have  not  increased  at  the  same  rate  it  may  be  necessary  to 
consider  each  separately.  If  actual  expenditure  for  equipment  still 
in  the  design  stage  will  take  place  in  the  future,  it  may  also  be 
necessary  to  consider  inflation  factors.  In  the  area  of  physical 
characteristics,  one  must  further  insure  that  such  common  terms  as 
weight,  speed,  and  distance  are  measured  in  like  units.  Often 
conversion  is  necessary  to  be  certain  that  all  elements  of  the  sample 
are  indeed  compatible. 

4.  Care  must  also  be  taken  to  insure  that  historicalycost  data, 
which  may  be  accumulated  from  several  sources,  truly  reflects  the 
actual  costs  incurred.  In  the  above  example,  for  instance,  accounting 
differences  among  studies  and  contractors  could  easily  result  in  wide 
variations  to  the  costs  actually  included  in  "tooling  costs". 

5.  Always  insure  that  your  answers  are  adjusted  for  sample  size. 

6.  Finally,  do  not  become  so  enamoured  with  an  estimating  model 
that  you  ignore  the  assumptions  made  in  its  development,  and  the 
reliability  of  the  sample  input  data.  A  computer  will  furnish  an 
impressive  and  detailed  readout,  even  if  the  input  data  is  unreliable. 
Carefully  scrutinize  sample  data,  data  sources,  and  assumptions  made 
in  developing  estimating  relationships. 
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LEARNING  CURVES 


In  concluding  this  section,  a  word  should  be  said  concerning  learning 
curves,  a  particular  type  of  parametric  relationships.  The  learning 
curve  supposition  is  based  upon  empirical  observations  that  manhour  labor 
inputs  in  the  production  process  seem  to  vary  inversely  with  cumulative 
output  produced  on  a  consistent  basis  (i.e,,  take  a  form  represented  by 
a  power  function).  However,  labor  productivity  is  affected  by 
technological  changes  as  well  as  alterations  in  the  level  of  utilization 
of  factors  of  production.  The  precise  manner  in  which  labor  inputs  per 
unit  of  output  change  as  a  result  of  all  these  factors  has  not  been 
formulated  or  generalized  for  the  Defense  industries. 

The  empirical  justification  for  a  power  function  learning  curve  as  a 
description  of  the  unit  labor  inputs  is  purely  statistical  in  nature. 

The  smaller  the  sample  size,  the  less  confidence  can  be  placed  in  the 
estimates  of  the  coefficients.  In  fact,  the  sample  size  may  be  so  small 
that  the  assumption  of  a  power  curve  relationship  may  be  clearly  unwarranted 
and  only  a  guess  by  the  estimator.  Whatever  the  sample  size  obtained, 
however,  the  resultant  sample  regression  must  be  used  with  care  for 
prediction  purposes.  No  probability  statement  can  be  made  about  a  point 
prediction  made  from  the  sample  data  in  relation  to  future  production. 
Probability  statements  can  be  made  only  for  interval  estimates. 

As  an  example  of  these  difficulties,  it  is  constructive  to  consider  a 
case  recently  examined  by  A.  R.  Gallant  and  appearing  in  the  December 
1968  issue  of  the  American  Statistical  Association  Journal.  The  Data 
in  Figure  7  gives  the  lot  size,  n*,  and  the  total  cost,  TCv,  respectively 
for  five  years.  From  this  information  the  average  cost  per  plane,  yv, 
for  each  lot  can  be  obtained.  Figure  8  gives  the  lot  plot  points  for 
three  alternative  methods  of  obtaining  lot  midpoints.  The  "Asher 
method"  refers  to  a  procedure  proposed  in  a  Rand  study  on  Cost/Quantity 
relations  by  Harold  Asher.  From  the  data  in  figure  8  least  squares 
were  computed.  Figure  9  presents  the  alternative  parameter  estimates 
for  alternative  methods  of  computing  lot  midpoints.  In  addition, 

Figure  9  presents  a  set  of  coefficient  estimates  which  were  generated 
by  Gallant  based  upon  a  probability  model  and  which  used  an  iterative 
procedure  for  solution. 

The  difference  in  slope  between  the  median  method  of  plotting  midpoints 
and  the  Gallant  iterative  procedure  is  0.73  percent.  In  addition,  the 


probability  estimates  allow  for  tests  of  hypotheses  and  confidence 
interval  statements.  For  the  particular  example  of  the  data  from 
Figure  7,  the  50  percent  and  90  percent  confidence  intervals  are 
(-.728,  -.617)  and  (-.842,  -.502)  or  in  terms  of  slope  (60.4%, 

65.2%),  and  (55.8%,  70.6%). 

To  see  the  effect  that  different  unit  curve  estimating  procedures 
might  have  on  cost  projections,  a  few  extrapolations  were  obtained 
from  the  previous  regression  curves.  Since  slope  values  and  tables 
of  learning  curve  values  are  almost  always  quoted  in  interger  values, 
Figure  10  shows  projections  for  the  Total  cost  of  alternative  buys 
of  F-4B  based  on  a  slope  value  of  62%  for  the  Median  and  Geometric 
Mean  methods  and  a  slope  value  of  63%  for  the  Asher  and  Gallant 
methods.  In  addition,  since  we  are  only  interested  in  the  relative 
differences  in  total  cost  extrapolations  and  not  their  absolute 
magnitude,  the  entries  in  the  table  are  expressed  as  percents 
of  the  Geometric  Mean  projections.  As  is  well  known,  the  nature 
of  a  power  learning  curve  is  such  that  significant  percentage 
variations  in  total  cost  can  result  from  much  less  significant 
percentage  changes  in  slope,  S.  As  is  evident  from  the  table,  a 
1%  difference  in  slope  may  yield  much  larger  percentage  differences 
in  total  cost  projections. 

In  addition  to  the  different  cost  estimates  generated  by  different 
least  squares  estimating  procedures,  further  uncertainty  concerning 
precise  cost  predictions  is  evidenced  by  the  90  percent  confidence 
interval  for  the  slope,  S,  derived  from  the  probability  model. 

During  concept  formulation  processes  for  new  systems,  there  would 
appear  to  be  no  sound  foundation  for  applying  the  learning  curve 
philosophy.  Thus  its  use  at  this  stage  of  cost  estimate  derivation 
should  be  made  only  after  an  in-depth  consideration  of  influencing 
factors.  Complete  justification  for  its  use  should  accompany  any 
such  cost  estimate. 

Moreover,  in  considering  the  general  usefulness  and  applicability 
of  learning  curves,  for  estimating  costs  at  such  times  as  system 
production  has  begun,  it  is  appropriate  to  reflect  on  the  conditions 
underlying  its  specification.  The  learning  curve  depicts  the 
relationship  between  labor  inputs  and  units  of  output  as  production 
affords  opportunities  to  "learn"  how  to  produce  more  efficiently. 
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F-4B  FLYAWAY  PRICES 


In  Millions  of  Dollars 

number  cost  per 


mi 

V 

purchased  (n*) 

lot  cost  (TC„) 

plane  (Y„) 

1958 

1 

3 

$  165.927 

$55,309 

1959 

2 

15 

156.981 

10.465 

1960 

3 

10 

62.438 

6.244 

1961 

4 

85 

320.238 

3.768 

1962 

5 

141 

327.834 

2.325 

254 

$1,033,418 

Figure  4-7 


ILLUSTRATION  OF  COMMON  METHODS 


cost  per  geometric 

lot  (v)  plane  (Y,r)  median  (X,.)  asher  (X,r)  mean 

1  $55,309  2.0  1.753  1.817 

2  10.465  11.0  9.691  10.040 

3  6.244  23.5  23.275  23.320 

4  3.768  71.0  64.691  66.360 

5  2.325  184.0  177.741  179.300 


(10)  Log  Yv  =  Log  a  -b  Log  Xv 

Figure  4-8 
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ALTERNATIVE  PARAMETER  ESTIMATES 


ESTIMATES 

MEDIAN 

ASHER 

GEOMETRIC  MEAN 

GALLANT 

a 

69.45 

63.16 

64.83 

61.59 

b 

-  0.6894 

-  0.6762 

-  0.6799 

-  0.6721 

s 

62.01% 

62.58% 

62.42% 

62.74% 

50  percent  confidence  interval: 

b: 

(-.728, 

-  .617) 

s: 

(60.4%, 

65.2%) 

90  percent  confidence  interval: 

b: 

(-.842, 

-.502) 

s: 

(55.8%, 

70.6%) 

C 


Figure  4-9 


4-25 


Total  Cost  of  np  Gallant 


Size  of 
Projected 
Buy  (np) 

Median 
Method  As 
Percent  of 

A 

GM 

Asher  Method 

As  A 

Percent  of  GM 

Units  in  Millions 
by  Geometric  Mean 
(GN)  Learning  Curve 

Method 
As  A  Per 
cent  of  i 

100 

107% 

111% 

126.254 

108% 

200 

107% 

111% 

229.812 

109% 

300 

107% 

112% 

318.641 

109% 

400 

107% 

112% 

397.009 

109% 

500 

107% 

112% 

'467.500 

109% 

SLOPE  (S)  FOR  GEOMETRIC  MEAN  AND  MEDIAN  METHODS  ROUNDED  OFF  TO  62 7, 

SLOPE  FOR  ASHER  AND  GALLANT  METHODS  ROUNDED  OFF  TO  63% 


Figure  4-10 


It  is  primarily  explained  in  terms  of  production  workers  repetition 
of  labor  operations  in  assembly  and  construction  work.  As  an 
empirical  description  of  reality,  it  is  desirable  to  consider  what 
this  implies  about  the  conditions  of  production.  The  Learning  Curve 
measures  the  affects  of  learning  on  labor  productivity  under  standard 
production  conditions.  Standard  production  conditions  means  that  each 
unit  of  output  is  produced  under  the  same  conditions  as  every  other 
unit  of  output.  The  learning  curve  is  constructed  and  used  under  the 
assumption  that  there  are  no  significant  effects  on  labor  productivity 
from  alterations  in  the  conditions  of  production  in  the  form  of  the 
following:  (a)  Alterations  in  the  rate  of  production;  (b)  Engineering 
modifications;  and,  (c)  Changes  in  the  level  of  technology.  The 
only  significant  factor  affecting  labor  productivity  has  to  do  with 
production  experience  accumulated  (and  the  increased  efficiency  this 
implies).  It  is  this  set  of  circumstances  which  underlies  the  use  of 
any  learning  curve.  It  must  be  recognized,  therefore,  that  when  the 
conditions  of  production  change,  the  applicability  of  the  learning 
curve  may  be  called  into  question. 

Engineering  Changes :  Engineering  changes  are  defined  here  as  all  of 
the  technological  alterations  in  the  product  which  is  under  production. 
If  engineering  changes  are  made  which  result  in  an  alteration  in  the 
technical  (performance)  characteristics  of  the  item  being  produced, 
it  is  only  significant  for  the  application  of  a  learning  curve  to  the 
extent  that  it  changes  the  conditions  of  production.  An  engineering 
change  may  be  minor  in  the  sense  that  while  it  affects  the  technical 
and  performance  characteristics  of  the  system,  it  does  not 
significantly  affect  labor  productivity  per  unit  of  output.  For 
example,  a  modification  of  the  avionics  portion  of  an  aircraft  system 
may  result  in  the  substitution  of  a  new  component  which  will  enhance 
the  performance  characteristics  of  the  aircraft.  It  will  not  affect 
the  learning  curve  nor  avionics,  however,  if  the  conditions  of 
production  are  not  changed.  This  may  be  true  even  when  the  cost  of 
the  new  component  over  the  old  may  significantly  alter  the  cost  of  the 
avionics  portion  of  the  system.  Therefore,  all  engineering  changes 
and  modifications  should  be  considered  in  light  of  how  they  affect 
the  conditions  of  production.  Engineering  modifications  may  result 
in  (a)  a  shift  in  the  Learning  Curve,  (b)  a  change  in  the  slope  of 
the  learning  curve  (rate  of  learning),  or,  (c)  some  combination  of 
the  preceeding  two  effects. 

Production  Rate  Changes:  Labor  productivity  will  also  be  affected 
by  the  degree  of  plant  capacity  utilization  and  the  costs  involved  in 
adjusting  highly  specific  labor  to  fluctuations  in  output  levels. 

The  costs  involved  in  hiring  and  training  a  firm's  labor  supply  may 
effectively  mean  that  the  firm  seeks  to  minimize  labor  losses  and 


turn-overs.  Consequently,  changes  in  a  firm's  production  rate  or 
capacity  utilization  may  be  met  with  smaller  fluctuations  (in  the  same 
direction)  in  the  labor  force.  A  decrease  in  the  rate  of  production 
which  is  not  met  with  a  corresponding  reduction  in  the  firm's  labor 
force  may  result  in  an  increase  in  labor  manhours  charged  against 
particular  units  or  lots  of  output.  Therefore,  the  cost  analyst  must 
investigate  the  affect  of  production  rate  changes  and  adjust  the  learning 
curve  if  necessary. 

Technological  Change:  A  firm's  method  of  production  may  change  over  time 
as  a  result  of  advances  in  technology  yielding  new  ways  of  producing 
products  more  efficiently.  Of  course,  labor  productivity  will  be  affected. 
For  example,  a  new  machine  may  allow  for  the  installation  and  assembly 
of  a  missile  motor  to  be  completed  with  fewer  manhours  of  labor  than 
previously.  This  means  that  labor  productivity  has  increased  as  a  result 
of  a  substitution  of  capital  for  labor.  Moreover,  the  change  in 
technology  may  alter  the  number  and  extent  of  the  opportunities  to  learn 
and  hence  may  affect  the  rate  of  learning  (slope  of  the  learning  curve). 

IMPLICATIONS  FOR  COST  ANALYSIS 

The  previous  considerations  imply  that  the  cost  analyst  has  a  degree  of 
leeway  in  the  exercise  and  use  of  learning  curves  for  any  particular  costing 
problem.  Moreover,  it  is  reasonable  to  expect  that  learning  curves  for  a 
particular  item  of  equipment  will  vary  according  to  the  type  model  and 
firm  producing.  Therefore,  a  cost  analyst  not  privy  to  contractor  data 
will  have  to  rely  either  on  acceptable  learning  curves  for  analogous 
items  of  equipment  or  an  industry-wide  average  learning  curve  as  a  rough 
approximation. 

A  useful  document  for  this  purpose  is  the  Defense  Contract  Audit  Agency's 
compilation  of  Learning  Curves.  The  pamphlet  is  entiled  "Report  on 
Improvement  Curve  Experience  (DCAAP  7641.14),  Revision  I,  April  1970. 

For  each  curve  there  is  listed  the  nomenclature  of  the  product,  the 
contractor  identification  code,  the  base  (dependent  variable),  the 
percentage  slope,  the  number  of  plot  points,  the  unit  numbers  included 
in  the  experience  data  and,  where  available,  the  estimated  percentages 
of  machine  and  assembly  work. 

All  of  the  curves  in  the  report  were  selected  on  the  basis  of  the 
following  criteria: 

1.  No  curve  was  accepted  where  the  slope  was  based  on  judgment, 
negotiation,  or  other  standards. 

2.  No  curve  was  accepted  where  significant  production  breaks  or 
engineering  changes  required  repositioning  plot  points. 
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3.  No  curve  was  accepted  with  less  than  a  .7  coefficient  of 
correlation  obtained  in  fitting  the  curve  to  historical  data. 

4.  No  curve  was  accepted  where  experienced  hourly  (or  cost)  data 
were  available  on  less  than  three  lots  or  units. 

Curve  slopes  contained  in  studies  such  as  these  should  be  used  by 
analysts  when  a  contractor's  records  are  not  available  for  the 
development  of  an  appropriate  curve. 

ESTIMATING  BY  ANALOGY 

The  method  of  analogy  is  based  on  direct  comparisons  with  historical 
information  on  like  or  similar  existing  systems  or  components.  It 
has  been  the  most  widely  used  method  of  analysis  to  date  although 
surely  not  the  most  accurate.  Thus,  the  current  emphasis  is  on  use 
of  parametric  techniques  as  previously  discussed.  The  major  drawback 
to  the  analogy  method  is  that  it  is  basically  a  judgment  process  and, 
as  a  consequence,  requires  considerable  experience  and  expertise  if  it 
is  to  be  done  successfully.  However,  there  are  occasions  when  the 
system  information  available  will  not  support  any  other  method  of 
analysis.  Figure  11  shows  the  relation  of  the  weapon  system  stage  of 
development  to  the  availability  of  data  and  to  the  methods  of  estimating 
used. 

The  analogue  technique  is  based  on  the  construction  of  relationships 
using  cost  data  from  logically  similar  systems.  I£j._s  used  when  there 
is  little  or  no  historical  information  available  on  the  specific  item 
and/or  when  (although  less  justified)  the  cost  estimate  is  required  so 
quickly  that  an  extensive  data  search  is  precluded.  In  the  latter 
cases,  it  is  a  choice  between  an  estimate  being  prepared  by  skilled 
cost  analysts  and  the  decision-maker  having  to  decide  without  any  cost 
data  available  to  him. 

There  are  two  types  of  analogues  which  may  be  used.  Similar  products 
can  be  compred  such  as  using  cost  data  on  commercial  vehicles  to 
estimate  costs  of  tactical  vehicles.  Secondly,  when  a  new  concept  or 
system  must  be  costed,  experience  gained  on  a  different  product  may 
be  used.  An  example  of  this  is  estimating  missile  costs  based  on 
aircraft  production  experience.  The  accuracy  of  this  type  estimate 
depends  on  the  experience  and  expertise  of  the  estimator  and  the 
availability  of  a  good  analogy.  Although  useful  for  making  rough 
comparison  checks,  this  approach  is  often  not  appropriate  for  use  by 
the  government  analyst  since  he  will  seldom  have  the  detailed  experience 
and  expertise  necessary  to  produce  a  sufficiently  accurate  estimate. 


RELATION  OF  WEAPON  SYSTEM  STAGE 
TO  INPUT  AND  ESTIMATING  METHODS 
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This,  combined  with  the  difficulty  of  determining  the  level  of 
uncertainty  in  the  estimate,  makes  the  analogy  approach  difficult  to 
apply  for  our  purposes. 

It  is  recommended  that  the  greatest  use  of  analogues  be  made  in 
conjunction  with  cost  estimates  made  parametrically.  If  resources 
permit,  estimates  can  be  made  parametrically,  by  engineering  methods 
and  by  analogy  and  all  three  can  be  comprared.  Your  most  expert 
analyst  should  make  the  analogy  estimate.  A  committee  can  then  review 
the  similarities  and  differences  in  the  three  estimates  and  choose  the 
best  one  based  on  the  amount  of  data  available  and  the  type  of  hardware 
being  considered. 

INDUSTRIAL  ENGINEERING  APPROACH 

The  industrial  engineering  approach  consists  of  a  consolidation  of 
estimates  from  various  separate  work  segments  into  a  total  project 
estimate.  As  an  example,  the  estimated  cost  of  production  of  a  new 
model  widget  consisting  of  work  contributions  from  10  separate  work 
divisions  in  a  plant  could  well  be  a  consolidation  of  10  separate  and 
detailed  estimates  each  of  which  may  be  composed  of  several  estimates 
itself.  Figure  12  shows  an  example  of  an  industrial  engineering 
estimate  of  the  cost  .of  a  steel  center  bracket.  Observe  the  amount  of 
detail  and  the  number  of  estimates  required  for  a  relatively  simple 
item. 

Estimating  by  engineering  methods  is  based  on  extensive  knowledge  of 
the  system  characteristics  and,  hence,  are  applicable  to  items  at  or 
near  the  production  stage  of  the  life  cycle.  It  is  necessary  for  the 
analyst  to  have  a  detailed  knowledge  of  the  system,  the  production 
processes,  and  the  production  organization.  In  using  the  engineering 
method,  the  system  or  item  of  hardware  is  broken  down  into  its  lower 
level  components  and  estimates  of  each  component  are  made.  Parametric 
methods  are  usually  used  in  estimating  the  costs  of  these  components, 
and  the  results  are  combined  with  estimates  of  the  costs  of  integrating 
the  components  to  arrive  at  a  total  system  cost.  An  advantage  to  this 
method  is  that  it  separates  the  parts  of  the  system  on  which  little 
data  is  available  or  which  are  new  technological  developments  and 
required  special  treatment  from  those  parts  which  can  be  treated  more 
conventionally.  Normally,  however,  the  method  has  several  drawbacks 
which  most  often  result  in  underestimation: 

1.  The  type  detail  required  is  not  available  to  a  government 
cost  analyst,  thus  making  this  approach  difficult  to  apply.  The 
approach  is  often  difficult  to  apply  even  at  the  corporate  level.  As 
an  example,  one  large  aerospace  firm  judges  that  the  use  of  this 
approach  to  estimate  the  cost  of  an  airframe  requires  more  than,  4000 
separate  estimates. 
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DETAILED  LABOR  COST  ESTIMATE  FOR  FORMING  A  STEEL  CENTER  BRACKET 
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2.  Inadequate  allowance  is  frequently  made  for  the  costs  of 
integrating  the  components  into  the  whole. 

3.  In  engineering  estimates,  some  production  labor  estimate®  ar«» 
made  as  percentages  of  the  detail  estimate  and  other  cost  elements  ;»re 
estimated  as  percentages  of  production  labor.  Thus  the  effect  of  a 
low  estimate  may  be  compounded. 

For  that  reason  some  firms  avoid  the  industrial  approach  when  possible. 
The  problem  arises  largely  because  each  individual  making  his  separate 
estimate  often  has  insufficient  information  available  to  make  a  reliable 
estimate,  and  little  means  to  evaluate  inherent  errors.  This  is 
especially  true  when  a  new  product  is  involved  and  the  estimator  must 
work  from  sketches,  blue  prints,  and  word  descriptions  of  an  item  which 
has  not  been  completely  designed.  In  these  cases  the  complexity  of  the 
work  can  be  easily  overestimated  or  underestimated.  Therefore,  a  cost 
estimate  combined  from  as  few  as  10  separate  estimates  also  combines 
the  errors  in  each  of  those  estimates  and,  in  aggregate  form,  there  is 
no  means  of  evaluating  the  errors  involved  or  the  level  of  uncertainty 
in  the  estimate.  For  these  reasons,  the  industrial  engineering  approach 
has  been  found  to  be  difficult  to  apply,  and  often  unreliable. 

As  in  the  section  on  "Estimating  by  Analogy"  it  is  recommended  that  a 
combination  of  methods  be  used.  The  primary  cost  estimate  should  be 
made  by  parametric  methods  early  in  the  life  cycle  and  may  be  made  by 
engineering  methods  as  the  system  nears  the  acquisition  stage. 

Secondary  estimates  can  then  be  made  by  use  of  other  methods.  If  a 
selection  of  the  "best"  estimate  cannot  be  made  by  involved  organizations 
then  the  various  estimates  with  their  rationales  should  be  presented 
to  the  decision  maker  and  he  can  select  the  "best"  for  the  system  under 
study. 

EXPERT  JUDGMENT 

The  analyst  may  appropriately  seek  the  judgment  of  recognized  experts 
at  all  phases  of  the  estimating  process  by  obtaining  the  advice  of 
either  in-house  or  contractor  experts.  The  necessity  of  using 
experienced  judgment  to  fill  in  gaps  in  data  has  long  been  recognized. 
However,  too  much  judgment  and  too  little  mathematical  analysis  is  often 
the  case.  The  keynote  in  using  judgment  must  be  reasonableness 
tempered  with  large  doses  of  impartiality.  Judgment  must  only  be 
employed  by  thoroughly  experienced  analysts  and  it  must  always  be 
identified  as  what  it  is  -  a  guess,  albeit  an  educated  guess. 

Expert  judgment  may  be  defined  as  the  comprehensive  knowledge  of  a 
system  by  an  individual  or  group  that  is  required  to  reach  a  conclusion 
that  is  not  directly  supported  by  data.  It  is  used  to  construct  most 
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CER's  particularly  to  check  the  behavior  of  the  relationship  when  it 
extends  significantly  beyond  the  data  base  or  where  the  data  base  is  too 
small  to  be  statistically  significant.  It  is  also  used  to  adjust  CER's, 
particularly  to  reflect  costs  of  another  technology.  An  example  of  this 
is  adjusting  a  CER  developed  for  a  mild  steel  structure  to  cost  an  item 
with  high  tensile  steel. 

The  use  of  expert  judgment  will  continue  to  be  necessary  at  times  but  it 
must  be  recognized  that  it  is  subject  to  large  inaccuracies.  The  analyst 
who  uses  expert  judgment  in, the  preparation  of  CER's  is  well  aware  of  its 
drawbacks  since  only  he  can  evaluate  the  limitations  of  the  data.  The 
best  use  of  expert  judgment  is  as  a  means  of  cross-checking  an  estimate. 

A  particularly  good  example  of  this  is  getting  the  expert  and  impartial 
judgment  of  a  consultant  to  check  what  is  felt  may  be  a  biased  in-house 
estimate. 

PROGRAM  STRETCHOUTS 

Program  stretchout  may  be  defined  as  an  increase  in  the  estimated  length 
of  time  required  to  place  a  hardware  system  into  the  hands  of  troops. 

This  stretchout  occurs  as  a  result  of  changes  primarily  within  the 
development  and  production  stages  of  the  system's  life  cycle.  Such 
stretchouts  may  be  the  result  of  either  risk  or  uncertainty  situations. 
Evaluation  of  this  stretchout  is  currently  handled  in  at  least  two 
separate  manners  and  either  may  be  considered  appropriate  depending  on 
the  situation  at  hand.  First,  within  the  development  phase  of  system 
production  the  procedures  of  the  Study  of  Trends  and  Escalation  of  Costs 
(STECPLOT)  are  applicable.  Through  this  method  the  influence  on  costs 
due  to  increases  in  the  time  required  to  complete  development  is 
ascertained. Second,  and  also  usable  within  the  development  phase  but 
adaptable  to  other  phases,  is  the  procedure  for  determining  the  monthly 
rate  of  fund  expenditure  (usually  during  the  peak  funding  requirement; 
e.g.,  SAM-D  estimate)  and  applying  this  rate  to  any  estimated  increase 
in  system  completion  time  or  system  milestone  intervals.  Third,  if  time 
can  be  developed  as  an  applicable,  separate  system  parameter  upon  which 
to  base  a  portion  of  total  system  cost,  then  program  stretchout  may  be 
assessed  during  the  parametric  estimating  process  during  the  concept 
formulation  stage  of  system  development.  The  adequacy  of  any  one  or 
combination  of  these  methods  is  essentially  a  matter  for  determination 
by  the  analyst  performing  the  estimate.  Research  should  be  done  to 
determine  if  there  is  a  preferential  method  for  treating  program 
stretchout  costing. 


1/  See  Chapter  10. 
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CHAPTER  5 


MEASURES  OF  ESTIMATE  VALIDITY 


UNCERTAINTY  AND  RISK 

The  disadvantages  of  decision-making  under  uncertainty  are  well-known. 
However,  the  fact  that  complete  knowledge  is  lacking  does  not  alleviate 
the  responsibility  for  using  whatever  knowledge  is  available  to  arrive 
at  a  decision. 

In  common  English  usage,  the  terms  "risk"  and  "uncertainty"  have  the 
same  connotation.  In  general,  "risk"  and  "uncertainty"  refer  to  the 
possibility  that  a  particular  characteristic  of  a  project  will  turn 
out  differently  than  forecasted  or  expected.  However,  in  economics 
and  decision-making  the  following  distinction  is  made  between  the  two 
words. 


A.  Risk  -  the  dispersion  of  the  probability  distribution  of  the 
event  whose  value  is  being  forecasted. 

B.  Uncertainty  -  the  measure  of  the  degree  of  lack  of  confidence 
that  the  probability  distribution  of  the  event  being  estimated  is  correct. 

Risk  then  says  that  the  value  of  an  event  cannot  be  estimated  with 
certainty  but  that  the  value  follows  a  certain  probability  distribution 
which  is  known.  Thus,  statistical  predictions  can  be  made  about  the 
particular  value  and  by  probability,  the  degree  of  confidence  one  has 
in  a  prediction  can  be  measured.  .Uncertainty  says  that  the  value  of  an 
event  cannot  be  estimated  with  certainty  and  that  the  value  may  follow 
a  probability  distribution  which  is  not  known  nor  certain. 

If,  under  risk,  a  probability  is  unity  or  one,  then  there  is  decision 
under  certainty.  Thus,  decision  under  certainty  can  be  classified  as 
a  special  case  of  decision  under  risk.  It  is  on  one  end  of  the  spectrum 
of  risk.  The  other  end,  of  course,  would  be  the  situation  where  there 
is  complete  ignorance  about  the  probabilities,  or  decision-making  under 
certainty.  As  one.  gains  more  information  about  a  situation,  there  would 
be  a  move  from  the  uncertain  into  the  risk  area. 

Example  1 :  Through  critical  component  sampling,  the  distribution  of  the 
shelf  life  of  a  missile  has  been  determined  as  shown  in  the  following 
table.  The  mean  is  4.0  years  and  one  can  forecast  that  the  shelf  life 
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of  a  missile  that  one  plars  to  build  that  has  similar  critical  components 
is  also  4  years. 


Distribution  of  Missile  Life 

7o  of  Missiles 

Life  (yrs)  With  Indicated  Life 


2.0  -  2.9 

10 

3.0  -  3.9 

37 

4.0  -  4.9 

41 

5.0  -  5.9 

11 

6.0  -  6.9 

1 

100 

Assuming  that  this  sample  of  shelf  life  represents  the  character  of  the 
underlying  population,  there  is  a  risk  of  47  in  100  that  the  shelf  life 
of  the  missile  will  be  less  than  4  years.  Or,  it  is  expected  that  47  out 
of  100  missiles  will  have  a  shelf  life  of  less  than  4  years. 

In  addition  to  this  risk,  it  is  uncertain  as  to  how  accurately  the  above 
sample  represents  the  probability  distribution  governing  the  population  of 
all  missiles  and  their  shelf-lives.  Therefore,  if  one  could  quantify  the 
degree  of  lack  of  confidence  that  one  has  in  the  sample  shelf  life 
distribution,  then  one  might  be  able  to  quantify  his  predictions  about 
shelf  life.  If,  for  example,  one  could  say  that  one  feels  90%  confident 
about  the  distribution,  then  the  statement  might  be  made  that  the  range  of 
probability  of  shelf  life  being  less  than  4  years  is  42.3  -  51.7  in  100. 
Risk  is  the  variation  from  what  is  expected  on  the  basis  of  the  predicted 
probability  distribution;  the  larger  the  variation  from  the  expected  value 
(average),  the  larger  the  risk.  Uncertainty  related  to  variation  from  the 
predicted  outcome  resulting  from  errors  in  determining  exactly  what  the 
probability  distribution  is. 

The  reason  why  the  differences  between  risk  and  uncertainty  have  been 
discussed  is  because  different  methods  of  problem  evaluation  apply  /to 
each.  Another  concept  that  is  important  to  risk  analysis  is  the  definition 
of  expected  value.  The  expected  value  of  an  alternative  is  the  average 
or  mean  value  that  the  alternative  would  have  in  the  long  run. 

Example  2:  The  shetf  life  from  example  1  showed  an  expected  or  average 
number  of  years  of  4.  This  means  that  if  the  analyst  selects  any 
missile,  it  is  expected  that  the  shelf  life  will  be  4  years;  or,  another 
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way  of  viewing  it,  upon  averaging  the  life  of  a  large  number  of 
missiles,  this  average  life  would  be  expected  to  be  4.  Consider  the 
following  situation: 


Costs 


Alternative  A 

Outcome  $ 

$20,000 

$40,000  $75,000 

Prob 

of  Outcome 

.70 

.20  .10 

Costs 

Alternative  B 

Outcome  $ 

$15,000 

$35,000  $100,000 

Prob 

of  Outcome 

.50 

.45  .05 

Expected  Cost  (Alternative  A)  ,70($20,000)  =  $14,000 

.20($40,000)  =  8,000 

. 10($75,000)  =  7,500 

$29,500 

Expected  Cost  (Alternative  B)  ,50($15,000)  =  $  7,500 

,45 ($3 5, 000)  =  15,750 
.05($100,000)=  5,000 

$28,250 

Given  the  above  information,  the  decision-maker  would  select  Alternative 
B  based  on  its  lower  expected  cost  (assuming  both  alternatives  have 
equal  effectiveness).  It  should  be  noted,  however,  that  while  the 
expected  value  for  Alternative  B  is  $28,250,  the  probability  of  the 
program  costing  $28,250  is  zero.  The  program  can  only  cost  $15,000, 
$35,000,  or  $100,000.  The  expected  value  must  be  understood  as  a  long 
run  average  such  that  if  the  same  program  was  conducted  a  very  large 
number  of  times,  say  1  million,  that  one  would  expect  that  500,000  times 
the  program  would  cost  $15,000,  450,000  times  to  cost  $35,000,  and 
50,000  times  to  cost  $100,000.  Upon  averaging  these  outcomes,  the 
average,  expected  value,  or  mean  would  be  $28,250.  However,  government 
programs  are  conducted  only  once  and  one  must  question  the  validity  of 
expected  value  alone  as  a  decision-making  measure. 

Example  3:  As  another  example  of  how  expected  value  can  be  misleading, 
consider  the  following  situation.  A  gambler  is  given  the  opportunity 
to  bet  $50,000  on  the  throw  of  a  dice.  If  the  dice  shows  a  6,  the 
bettor  wins  1  million  dollars;  if  any  other  number  comes  up,  he  loses 
his  $50,000.  Thus,  there  are  two  alternatives: 

Alternative  1  -  Keep  the  $50,000 
Expected  value  =  $50,000 

Alternative  2  -  Bet  the  $50,000 

Expected  value  =  1/6  (1,000,000)  -  5/6  (50,000)  =  166,667  - 
41,667  =  $125,000 
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Alternative  2  has  the  higher  expected  value  and  the  dice  is  thrown  once. 
What  is  the  decision? 

A  millionnaire  may  accept  this  wager  because  $50,000  is  a  small  percent  of 
his  fortune.  However,  most  of  us  would  not  accept  the  bet.  The  point 
illustrated  here  is  that  decision-making  with  expected  cost  as  the  criteria 
is  valid  only  when  the  maximum  loss  that  it  is  possible  to  incur  is  not 
so  great  as  to  seriously  affect  the  well-being  of  the  spender. 

Therefore,  considering  Alternatives  A  and  B  mentioned  in  example  2,  even 
though  B  had  a  lower  expected  cost,  there  was  the  possibility  under  B 
for  the  program  to  cost  as  much  as  $100,000.  Suppose  Congress  told  the 
Army  that  if  this  program  exceeded  $80,000  in  expenses,  the  Army  budget 
would  be  cut  in  half  next  fiscal  year.  The  Army  probably  would  not  be 
willing  to  take  that  57.  chance  of  Alternative  B  costing  $100,000  for 
possible  consequences  such  as  this.  They  would  probably  choose  Alternative 
A  even  though  expected  cost  is  higher. 

It  has  been  shown  that  cons!1,  '  ;a  >.  expected  value  alone  does  not 
always  lead  to  the  correct  decision,  wtho  factors  may  also  be  relevant 
to  a  decision. 

Example  4: 

Project  R  Project  S 


Cost 

Prob 

Cost 

Prob 

$200,000 

.10 

$160,000 

.10 

$220,000 

.20 

$200,000 

.20 

$240,000 

.40 

$240,000 

.40 

$260,000 

.20 

$280,000 

.20 

$280,000 

.10 

$320,000 

.10 

The  expected  costs  of  each  alternative  is  the  same  yet  the  dispersion 
about  the  expected  cost  of  Project  S  is  greater.  This  dispersion  in 
economics  is  the  quantification  of  risk.  Therefore,  Project  S  is  said 
to  have  greater  risk  than  Project  R  and  one  would  be  more  likely  to 
select  Project  R. 


Graphically: 


Probability 


.40 

.30 

.20 

.10 


Project  R 


iOo 


2te 


2I«> 
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What  is  illustrated  here  is  the  concept  of  risk  as  the  dispersion  about 
the  expected  value  or  average.  The  greater  the  dispersion,  the 
greater  the  risk.  In  general,  it  could  be  said  that  for  two  alternatives 
with  equal  expected  values,  the  one  with  the  smallest  risk  (least 
dispersion  from  the  expected  value)  is  usually  the  most  desirable. 

Consider  the  following  example: 

Example  5: 


Project  J 


Project  K 


Cost 

Prob 

$220,000 

.10 

$230,000 

.20 

$240,000 

.40 

$250,000 

.20 

$260,000 

.1-6 

Exp  cost 

$240,000 

Cost 

Prob 

$100,000 

.10 

$150,000 

.20 

$200,000 

.40 

$250,000 

.20 

$300,000 

.10 

Exp  cost  $200,000 


Probability 


Project  K  has  a  lower  expected  cost  than  does  Project  J;  however,  it 
has  greater  risk  involved  because  of  the  wider  dispersion  from  the 
mean.  A  decision  under  these  conditions  is  a  difficult  one  and  would 
depend  on  more  information  such  as  relative  preferences  for  cost 
saving  versus  risk,  the  consequences  of  the  program  exceeding  a 
particular  cost,  or  other  factors  bearing  on  the  acceptability  of 
various  outcomes. 


To  this  point,  the  discussion  has  only  been  on  discrete  or  point 
distributions  of  costs.  Theoretically,  one  can  look  at  costs  that 
have  continuous  distributions. 


Example  6: 


Probab. 


The  total  area  under  the  continuous  probability  curve  is  equal  to  1  because 
all  the  possible  outcomes  are  included.  What  the  curve  says  is  that  if 
x  is  a  cost,  and  the  area  between  a  and  b  is  1  and  the  area  between  a  and 
x  is,  say,  .75,  then  the  probability  that  the  cost  will  turn  out  less 
than  x  is  .75;  or  conversely,  the  probability  that  the  cost  will  be  more 
than  x  is  .25.  Also,  the  probability  that  cost  will  be  mere  than  b  is 
0,  more  than  a,  1. 


In  both  discrete  and  continuous  distributions,  risk  as  defined  as 
dispersion  from  the  expected  value  and  can  be  measured  by  using  a  common 
statistical  measure  called  the  variance,  or  its  square  root,  the  standard 
deviation.  The  standard  deviation,  <T  ,  has  the  following  mathematical 
definition: 

CT  = 


Example  7 : 

Alternative  1 


Costs 
$  2 
$  5 
$  7 
$10 


Probability 

.3 

.4 

.2  Expected  Value  =  $5.00 

.1 

J  U-5)*  +  (S-t>/*  +  a-YlztXl£>-fl*  e  3  ( 


Costs 


Alternative  2 

Probability 

.3 

.4 

.2 

.1 


Expected  Value  =  $5.00 


5-6 


It  is  obvious  from  examination  of  the  Alternatives  that  Alt.  2  has 
the  greater  risk  (dispersion)  than  1;  and  the  calculation  of  the 
respective  standard  deviations  bears  out  this  fact. 

With  continuous  distributions,  the  concept  is  the  same,  the  calculation 
is  theoretically  the  same  but  since  the  distributions  are  continuous, 
the  technique  of  integration  must  be  used. 

Continuous  distributions  can  be  shown  as  follows: 


As  in  discrete  distributions,  they  can  be  evaluated  as  to  which 
decision  should  be  made  on  the  basis  of  risk  and  expected  value. 

The  one  case  which  was  difficult  to  evaluate  was  the  one  in  which  one 
alternative  had  a  greater  expected  cost  than  the  other;  but  had  3 
lesser  risk  attached  to  it.  A  possible  method  of  evaluating  such 
alternatives  is  to  attach  an  "evaluator”  to  the  standard  deviation 
(risk  factor)  and  take  the  sum  of  the  expected  cost  plus  the  standard 
deviation  times  its  evaluator.  Whichever  has  the  lowest  statistic 
gets  the  decision. 

Example  8:  Consider  Project  J  and  Project  K  of  example  5.  Project  K 
had  less  an  expected  cost  but  greater  dispersion. 

Project  J  Project  K 

Expected  Cost  $240,000  Expected  Cost  $200,000 

Standard  Deviation  14,140  Standard  Deviation  70,700 

Decision  Statistic:  Project  J:  240,000  +  e  (14,140) 

Project  K:  200,000  +  e  (70,700) 

If  e  is  chosen  to  be  approximately  .71,  then  the  statistic  for  both 
is  equal.  Therefore,  our  decision  criteria  would  be  as  follows: 

e  >  .71  Choose  Project  J 

e  =  .71  Either 

e^  .71  Choose  Project  K 

The  evaluator  "e"  would  have  to  be  chosen  subjectively  and  would 
depend  on  other  factors  such  as  time  to  complete  program,  relative 
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performance,  technical  risk,  etc.  It  should  be  obvious  that  if  the 
upper  cost  range  of  the  project  with  greater  risk  but  lower  cost  is 
lower  than  the  expected  cost  of  the  project  with  less  risk,  the  former 
should  be  selected. 

Let  us  now  consider  a  case  of  how  the  concepts  just  explained  can  be 
used  in  a  hypothetical  situation. 

Example  9:  The  Army  plans  to  build  a  particular  piece  of  equipment  to 
perform  a  certain  mission.  The  design  engineers  have  developed  two 
configurations  that  can  be  broken  down  into  various  components.  At 
this  point,  the  cost  people  in  conjunction  with  the  engineers  estimate 
the  most  likely  costs  of  each  of  the  components  for  each  design;  and, 
in  addition,  estimate  an  upper  and  lower  range  for  each  of  these  most 
likely  values.  The  results  can  be  put  in  hypothetical  tabular  form  as 
follows: 

($000) 


Configuration  A 

Most 

Configuration  B 

Most 

Likely 

Likely 

Cost 

Range 

Cost 

Range 

Component  a. 

$12 

$10-15 

Component  b^ 

$31 

$26-35 

a2 

22 

18-25 

b2 

10 

9-11 

a3 

6 

4-  8 

b3 

8 

5-10 

a4 

5 

4-  7 

b4 

9 

7-10 

Total  Cost 

$55 

$44-67 

Total  Cost 

$58 

$48-66 

At  this  point,  two  important  points  must  be  noted: 

1.  The  above  costs  are  assumed  to  be  random  costs  that  approxi¬ 
mately  follow  a  Beta  distribution  (Based  on  a  study  on  costs). 

2.  Central  Limit  Theorem  (CLT)  says  "a  new  variable  formed  by  the 
addition  of  independently  distributed  variables  follows  a  normal  distri¬ 
bution  and  its  mean  is  the  sum  of  the  means  of  the  component  distributions 
and  its  variance  is  the  sum  of  the  variances  of  the  component  distributions." 
(NOTE:  More  than  4  or  5  independent  distributions  should  be  summed  - 

30  would  be  more  suitable.) 

To  approximate  the  mean  (expected  costs)  and  variance  of  each  component, 
the  following  formulae  for  Beta  distributed  random  variables  will  be  used. 

Mean  =  A  +  4M  +  B 
6 

Variance  =  / B-A) 2  0  =  B-A 

\  6)  6 
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Where  A  =  lower  bound 
B  =  upper  bound 
M  =  Most  likely  costs  (Mode) 

The  following  table  is  thereby  constructed: 


Configuration 

_A 

Mean 

Std.  Dev. 

Configuration 

_B 

Mean 

Std.  Dev. 

Component  a^ 

12.2 

.8 

Component  b. 

30.8 

1.5 

a2 

21.8 

1.2 

b2 

10.0 

.3 

a3 

6.0 

.7 

b3 

7.8 

.8 

34 

5.2 

.5 

b4 

8.2 

.5 

a5 

Total 

10.0 

55.2 

.7 

3.7 

Total 

56.8 

3.1 

By  the  central  limit  theorem,  the  total  costs  for  each  configuration 
approximates  the  normal  distribution.  Configuration  A  has  a  mean 
or  expected  cost  $55,200  and  standard  deviation  of  $3,700; 
configuration  B  has  mean  of  $56,800  and  standard  deviation  of  $3,100. 
A  has  smaller  expected  value  but  larger  risk.  Plotting  the  normal 
distribution  for  each  of  these  configurations,  the  following  graphs 
are  drawn: 


Alt.  A 
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A 


The  properties  of  normal  distributions  are  such  that  the  area  between 
the  bounds  of  1  standard  deviation  to  either  side  of  the  mean  is  about 
687o  of  total;  between  2  standard  deviations,  about  95%,  between  3 
standard  deviations,  about  99.9%.  What  this  means  is  that  for 
configuration  A,  the  probability  of  cost  being  less  than  $66,300 
(3  standard  deviations  from  the  mean)  is  almost  100%  (99.95%).  In 
this  case,  it  would  be  difficult  to  choose  between  the  two  configurations 
as  relates  to  costs  and  cost  risk,  and  other  considerations  would  have 
to  be  brought  to  bear  on  the  decision. 

In  another  situation  of  this  type,  quantification  of  statistical  cost 
risk  would  be  carried  out  in  a  similar  manner.  One  would  consider  the 
costs  of  components,  sub-components,  etc.,  estimate  the  most  likely 
costs  and  ranges,  sum  up  to  the  total  cost,  and  using  the  CLT  assume  the 
total  cost  is  approximately  distributed  according  to  a  normal  distribution 
with  mean  and  variance  as  obtained  as  shown.  As  a  result,  one  could 
make  statistical  risk  predictions  about  the  costs  of  the  particular  system 
being  estimated.  Of  course,  in  the  example  used  the  results  were  based 
on  the  correctness  of  the  assumed  probability  distribution.  Uncertainty 
over  the  kind  of  distribution  underlying  cost  estimates  would  invalidate 
quantitative  predictions  about  risk. 

It  has  been  shown  how  cost  risk  may  be  quantified  and  evaluated  in  terms 
of  statistics.  As  mentioned,  the  basic  assumption  of  risk  analysis  is 
that  the  probability  distribution  is  known.  In  most  practical  situations, 
roulette  excluded,  the  probability  distribution  of  events  is  not  known 
for  certain.  Therefore,  other  methods  of  evaluation  to  deal  with 
uncertainty  have  been  developed.  Unfortunately,  a  great  amount  of 
subjectivity  is  required  and  use  of  different  methods  will  not  yield 
consistent  decisions. 

In  the  situation  where  the  probability  of  future  outcomes  is  not  known 
for  certain,  intuition  would  tell  us  to  try  to  subjectively  estimate 
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the  probabilities  of  the  outcomes  for  each  alternative.  Then  using 
the  ideas  of  risk  analysis,  a  decision  may  be  made.  This  technique 
is  called  the  Bayes  Decision  Rule. 

Example  10;  Outcome  (Costs) 

S1  S2  S3 


Alternatives  2  4  6 


A2  4  1  5 


It  is  estimated  that  will  occur  with  probability  =  P^  =  .7,  S2 
with  P2  =  .2  and  S3  with  P3  =  .1.  Using  expected  value  as  the 
decision  criterion 


Exp  (A.)  =  .7(2)  +  .2(4)  +  .1(6)  =  2.8 
Exp  (A2)  =  .7(4)  +  .2(1)  +  .1(5)  =  3.5 

Based  on  expected  value  criterion  choose  Aj  (lowest  cost).  It  is,  of 
course,  not  certain  that  the  probabilities  assumed  are  true.  Varying 
the  probabilities  and  outcomes  leads  to  simulation  and  sensitivity 
analysis,  which  is  discussed  later  in  this  chapter. 

A  second  technique  used  to  cope  with  uncertainty  is  called  the  LaPlace 
Principle.  The  LaPlace  Principle  assumes  that  all  outcomes  have 
equally  likely  chances  to  occur.  Again,  compute  expected  value  and/or 
standard  deviation  and  evaluate. 


Example  11:  Using  the  same  alternatives  and  outcomes  as  in  example  10, 
assume  the  probability  of  each  outcome  is  1/3.  Then:  Exp  (Ai  )-12=4 

3 


Select  A2  for  lowest  expected  cost. 


Exp  (A2 )=10=3 . 3 
3 


Another  method  used  is  called  the  Minimax  Criterion.  This  method  is 
used  when  complete  uncertainty  about  the  probability  distribution  exists 
and  subjective  estimates  are  impossible.  The  Minimax  Criterion  simply 
says  "examine  the  highest  (maximum)  cost  that  can  occur  for  each 
alternative;  and  select  the  alternative  that  has  the  lowest  (minimum) 
of  these  costs."  The  philosophy  is  pessimistic  and  attempts  to  set  an 
upper  bound  on  costs. 
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Example  12: 

Outcomes 

(Costs) 

S1 

s2 

S3 

A1 

4 

1 

3 

Alternatives 

A2 

1 

1 

5 

A3 

2 

3 

3 

Maximum 

Costs 

4 

5 
3 


Alternative  A3  has  the  lowest  of  the  maximum  costs  under  this  criterion 
and  would  be  chosen. 


Similar  to  the  Minimax  Criterion  is  a  fourth  technique  called  the 
Minimum  Criterion.  This  technique  is  used  if  the  analyst  is  optimistic. 
Essentially,  the  minimum  cost  for  each  alternative  is  examined;  and  the 
alternative  with  the  lowest  (minimum)  of  these  minimums  is  chosen. 


Example  13:  Outcomes  (Costs) 


Si 

s2 

S3 

Minimum 

A1 

4 

1 

3 

Costs 

Alternatives  A2 

1 

1 

5 

1 

a3 

2 

3 

3 

1 

2 


Select  either  Aj  or  A2  using  minimum  as  the  criterion. 


Still  another  technique,  exotically  named  the  Minimax  Regret  Criterion, 
can  be  employed.  If  one  alternative  is  chosen  and  another  alternative 
would  have  given  us  a  lower  cost  under  the  outcome  that  actually  occurs, 
one  would  regret  the  fact  that  the  costs  could  have  been  lower.  The 
maximum  potential  regret  can  be  measured  by  subtracting  the  estimated 
cost  for  each  alternative  under  each  outcome  from  the  lowest  estimated 
cost  for  each  alternative  under  the  given  outcome.  The  analyst  then 
lists  the  largest  potential  regret  for  each  alternative.  Finally  the 
analyst  chooses  the  alternative  with  smallest  potential  regret. 


Example  14; 


Alternatives 


Outcomes  (Costs) 
S1  S2  S3 

A1  4  1  ~ 

a2  115 
a3  2  3  3 


Regret  Matrix 

S1  S2  S3 

-3  0  ~  — (T  ~ 

0  0-2  -2 

-1  -2  0  -2 


Choose  A2  or  A3  since  they  show  (in  absolute  value)  the  minimum  amount 
of  regret. 
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To  again  illustrate  the  techniques  just  presented  and  also  to  show  how 
each  of  these  different  criteria  can  lead  to  different  decisions, 
consider  the  following  example: 


Example  15: 

Si 

s2 

S3 

S4 

Bayes* 

LaPlace 

Mini 

Max 

Mini 

Min 

Regret 

A1 

2 

3 

2 

4 

2.9 

2.75 

4 

2 

O 

-  L. 

Alternatives 

a2 

3 

1 

2 

5 

2.8 

2.75 

5 

1 

-3 

A3 

5 

2 

2 

2 

2.6 

2.75 

5 

2 

-3 

A4 

2 

2 

1 

5 

2.7 

2.50 

5 

1 

-3 

*  Assumes  for  Bayes  =  .2,  P2  =  .3,  P3  =  .2,  P4  =  .3 

1)  Bayes  says  select  A2 

2)  LaPlace  says  select  A^ 

3)  Mini  Max  says  select  A^ 

4)  Minimum  says  select  A2  or  A4 

5)  Regret  says  select  A4 

The  disadvantages  of  making  decisions  under  uncertain  conditions  are 
obvious.  Depending  upon  one's  subjective  judgment,  a  different  decision 
can  be  made  under  each  criteria. 

It  will  be  advantageous  to  review  some  earlier  definitions  at  this 
point.  In  economics  and  decision  theory,  a  rigorous  definition  of 
uncertainty  is  desired.  Uncertainty  attempts  to  measure  the  degree 
of  confidence  the  analyst  has  that  the  probability  distribution  chosen 
is  correct  for  the  event  being  estimated.  The  value  of  the  event  cannot 
be  estimated  exactly  and  this  ^alue  follows  a  probability  distribution 
not  known  for  certain.  It  is  the  variation  which  is  found  in  the 
probability  distribution  and  caused  by  errors  in  forecasting  the 
precise  nature  of  the  actual  probability  distribution.  On  the  other 
hand,  the  rigorous  definition  of  risk  says  that  it  is  possible  to 
describe  all  the  outcomes  of  the  situation  and  to  assign  meaningful 
objective  numerical  probability  weights  to  each  such  outcome.  The 
value  of  the  event  cannot  be  estimated  exactly  but  under  risk  the  value 
follows  a  particular  probability  distribution  which  is  known. 

In  practice,  it  is  not  usually  realistic  for  cost  analysis  to  follow 
these  rigorous  definitions  and  differentiations  at  the  current  time. 


It  is  possible  to  use  such  techniques  as  the  Bayes  Decision  Rule,  the 
LaPlace  Principle,  the  Minimax  Criterion,  etc.  to  evaluate  "uncertainty" 
as  defined  above;  however,  the  use  of  such  techniques  is  not  recommended 
because  they  are  too  highly  dependent  on  judgment  and  particularly 
because  the  use  of  different  techniques  can  lead  to  different  decisions. 
In  most  cases,  it  is  necessary  to  translate  a  problem  of  decision¬ 
making  under  uncertainty  to  a  problem  of  decision-making  under  risk. 

See  Figure  1  for  a  general  approach  which  bounds  uncertainty  and 
attempts  to  make  it  more  specific.  The  analyst  thus  has  three  choices: 

1.  Treat  the  decision  as  one  under  uncertainty  -  this  course  is 
not  recommended. 

2.  Acquire  data  or  perform  experiments  so  that  the  decision  is 
translated  to  one  under  risk.  This  course  is  often  blocked  by  such 
practicalities  as  lack  of  time  and  money. 

3.  Transform  the  decision  to  one  under  risk  by  obtaining 
probabilities  subjectively  assigned  by  expert  judgment.  It  is  recognized 
that  this  course  introduces  further  uncertainty  but  it  may  be  and  often 
is  the  only  course  available.  Such  judgment  must  be  based  on  past 
experience  with  systems  similar  to  the  one  being  costed.  It  is  also  to 
be  emphasized  that  the  basis  for  the  judgment  must  be  fully  documented 
and  made  a  permanent  part  of  the  cost  estimate. 

It  is  also  recommended  that  uncertainty  be  quantified  as  objectively 
as  possible  in  all  cases.  All  cost  estimates  should  include  a  section 
giving  the  point  estimate  (most  likely  value)  and  a  probable  high-low 
range  for  the  estimate  with  a  90%  confidence  level.  That  is  the 
probability  that  the  actual  system  cost  will  be  equal  to  or  greater 
than  the  high  value  is  5%  and  the  probability  that  the  cost  will  be 
equal  to  or  lower  than  the  low  value  is  5%.  A  qualitative  paragraph 
may  also  be  desirable  but  it  should  not  be  the  only  uncertainty 
evaluation  offered. 

The  level  of  detail  an  uncertainty  evaluation  presents  is  also  a 
matter  of  judgment.  Generally,  aggregated  cost  uncertainties  should 
be  presented  in  RDTE,  recurring  investment,  non-recurring  investment, 
and  operating  cost  categories.  However,  early  in  the  life  cycle  it 
may  be  realistic  to  only  present  a  system  uncertainty  estimate  and 
range.  Later  in  the  development  cycle,  it  may  be  valuable  to  also 
highlight  the  uncertainty  associated  with  a  particular  cost  element. 

An  overall  system  uncertainty  evaluation  should  also  be  included. 
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APPROACHES  TO  UNCERTAINTY 


CONCEPT  OF  A  DECISION 
UNDER  UNCERTAINTY 


CHOICE 


OBTAIN  EXPLICIT  DATA 

AS  A  BASIS  FOR 
PROBABILITIES 

• 

...  - 

DECISION 

P 

UNDER 

RISK 

OBTAIN  EXPERT  JUDGMENT 

AS  A  BASIS  FOR 

SUBJECTIVE  PROBABILITIES 

Figure  5-1 
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1/  2/ 

This  may  bu  arrived  at  using  Monte  Carlo-  or  asymmetric—  estimating 
techniques. 

The  use  of  standard  statistical  techniques  (confidence  intervals, 
t-test  of  significance  of  the  standard  error,  etc.)  to  deal  with 
uncertainty  often  cannot  be  justified.  Usually  the  sample  size  is  too 
small  and  a  normal  distribution  for  the  underlying  population  cannot 
be  assumed.  However,  the  standard  deviation  may  be  useful  measure  of 
uncertainty  if  there  are  five  or  more  data  points.  When  CER's  are  used 
in  the  estimating  process,  the  uncertainty  section  should  definitely 
include  a  standard  error  of  the  estimate. 

The  value  of  the  Monte  Carlo  technique  has  not  yet  been  evaluated 
in  fact.  It  seems  to  be  valuable  provided  the  analyst  has  some 
confidence  in  the  validity  of  his  inputs  and  provided  he  has  extensive 
computer  time  available.  There  are  also  hand-computational  techniques 
for  determining  cost  ranges  whether  the  distribution  is  symmetric  or 
asymmetric. 

Sensitivity  analysis  has  been  cited  as  a  valuable  technique  for 
uncertainty  evaluation.  It  is  useful  in  identifying  the  cost  sensitive 
system  elements  but  does  not  produce  a  set  quantitative  measure  of  un¬ 
certainty.  This  issue  is  further  discussed  in  a  succeeding  section 
of  this  chapter  dealing  with  sensitivity  analysis. 

In  carrying  out  an  estimate  under  conditions  of  great  uncertainty,  it 
may  be  desirable  to  recommend  special  tests  or  other  methods  in  order 
to  define  the  estimating  problem  in  terms  of  quantifiable  risk,  and, 
in  any  study  addressing  uncertainty,  the  presentation  of  the  analysts' 
qualitative  evaluation  of  the  following  four  topics  is  of  particular 
importance: 

1.  How  much  uncertainty  exists. 

2.  What  steps  can  be  taken  to  reduce  uncertainty. 

3.  How  much  will  it  cost  to  reduce  uncertainty. 

4.  How  much  uncertainty  will  be  reduced  as  development  progresses. 


1/  "A  Monte  Carlo  Simulation  Approach  to  Cost  Uncertainty  Analysis," 

M.  F.  Gutowski,  et  al,  RAC-TP-349,  Research  Analysis  Corporation,  1969. 

2/  "Adding  Costs  Est.  that  are  not  Symmetric  About  the  Most  Likely 
Value,"  Sutherland,  Draft  RAC-CR-23,  Research  Analysis  Corporation,  1970. 
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The  reduction  of  uncertainty  by  acquiring  knowledge  occurs  as 
development  progresses,  and  the  fact  that  development  produces 
information  about  costs,  about  performance,  and  about  availability 
dates  has  important  implications  for  the  conduct  of  cost  analysis. 

It  is  now  of  key  interest  to  decision  makers  whether  it  would  pay  to 
buy  more  information;  that  is,  to  undertake  development  up  to  some 
later  stage,  before  a  final  choice  is  made. 

As  an  example  of  this  situation  consider  a  problem  of  deciding  on  a 
course  of  action  with  respect  to  a  future  ICBM  capability.  There  are 
two  promising  alternatives:  (1)  Missile  A,  which  uses  a  storable 
liquid  propellant,  and  (2)  Missile  B  which  uses  a  solid  propellant. 

To  simplify  things  assume  that  the  operational  dates  of  the  system 
and  the  capability  that  must  be  available  are  fixed.  What  is  wanted 
then,  is  to  achieve  the  given  capability  during  the  given  time  period 
at  a  minimum  cost--where  cost  is  defined  to  include  all  costs  of 
development,  procurement,  and  operation.  When  confronted  with  this 
problem,  the  decision  maker  is,  of  course,  uncertain  about  which  of  the 
two  alternatives  will  in  fact  yield  the  minimum  cost.  Assume,  for 
the  sake  of  a  simple  example,  that  his  best  judgment  about  probable 
costs  of  Missile  A  and  Missile  B,  based  on  the  information  available 
to  him,  is  summarized  in  Figure  2.  This  figure,  in  other  words, 
portrays  his  personal  estimates  of  the  probable  costs  of  these  two 
missile  systems. 

In  brief,  our  decision  maker  is  convinced  that  Missile  A  will  cost 
either  $8  billion  or  $10  billion  and  that  these  two  are  equally 
probable.  Similarly,  he  is  convinced  that  Missile  B  will  cost  either 
$7.5  billion  or  $13.5  billion,  once  again  with  equal  probability. 

Probability 


Mean  or  #7.0  *10.5 

Expected  Value 

Fig.  5-2  Probable  costs  of  Missiles  A  and  B 
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Given  the  information  shown  in  Fig.  2,  the  decision  maker  might  simply 
assume  that  the  rational  choice  is  to  select  the  system  with  the  lowest 
expected  cost.  In  other  words,  he  would  choose  Missile  A  on  the  grounds 
that  if  he  makes  such  decisions  repeatedly  he  would  expect  an  average 
cost  of  $9.0  billion  for  Missile  A  as  opposed  to  $10.5  billion  for  Missile 
B. 


Given  our  decision  maker's  estimates  of  probable  costs,  there  are  four 
possible  states  of  the  real  world  that  can  exist.  These  are  shown  in 
Figure  3.  The  figures  in  the  table  represent  the  four  possible 
combinations  of  costs  for  Missile  A  and  Missile  B  which  could  occur, 
given  the  probable  cost  estimates  shown  in  Figure  2. 

Figure  5-3 


State  of  the  world 

Cost  of  Missile  A 
(billions  of  $) 

Cost  of  Missile  B 
(billions  of  $) 

Si 

8.0 

13.5 

s2 

10.0 

13.5 

S3 

8.0 

7.5 

S4 

10.0 

7.5 

When  our  decision  maker  is  first  confronted  with  the  problem,  of 
course,  he  does  not  know  which  of  these  four  situations  is  in  fact 
true.  If  he  chooses  Missile  A,  that  is,  if  he  chooses  on  the  basis 
of  expected  costs,  and  if  either  or  S2  is  the  true  state  of  the 
world,  the  choice  of  Missile  A  will  have  been  best.  But  if  he  chooses 
Missile  A,  and  either  S3  or  S4  is  the  true  state  of  the  world,  then 
he  has  chosen  wrongly. 

He  would  very  much  like  to  avoid  choosing  wrongly,  but  if  he  cannot 
get  more  information  as  a  basis  for  his  decision,  he  cannot  improve  on 
a  straightforward  choice  of  Missile  A.  It  is  at  this  point  that  the 
second  observation  cited  earlier  about  the  nature  of  development 
becomes  relevant.  The  reason  we  undertake  development  is  precisely  to 
get  information--information  that  will  reduce  uncertainty.  If  our 
decision  maker  has  an  opportunity  to  improve  on  the  reliability  of 
his  cost  estimates  through  partial  development,  he  may  then  be  able 
to  improve  on  the  results  he  would  get  by  simply  choosing  Missile  A. 

To  illustrate  this,  take  the  extreme  case  and  see  how  much  it  would 
be  worth  to  find  out  with  certainty  which  of  the  four  situations  in 
Figure  3  is  the  true  state  of  affairs.  Assuming  that  Missile  A  is  the 
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best  choice  in  the  absence  of  such  knowledge,  it  is  now  asked  how  much 
the  decision  maker  would  be  willing  to  pay  to  determine  which  of  these 
four  states  will  actually  exist.  In  order  to  answer  this  question 
let  us  first  suppose  he  is  told  which  of  the  four  will  prevail.  When 
he  is  told  that  either  Sj  or  S2  is  true,  he  will  choose  Missile  A  just 

as  he  did  before,  because  this  will  give  him  the  lowest  cost.  But, 

when  he  is  told  that  S3  or  S4  is  true  he  would  choose  Missile  B  in 
order  to  minimize  costs.  Naturally,  at  the  time  he  has  to  make  his 
decision,  he  never  really  knows  which  of  the  four  will  turn  out  to  be 
true.  In  fact,  he  regards  each  as  equally  likely — to  each  he  has 
assigned  a  probability  of  0.25  that  it  will  be  the  one  that  actually 
exists.  Therefore,  his  average  or  expected  savings  if  B  were  chosen 
rather  than  A  is  0.25  x  $0.5  billion  =  $0,125  billion  with  respect  to 
state  S3  plus  0.25  x  $2.5  billion  *  $0,625  billion  with  respect  to 
state  S4,  or  $0.75  billion  in  all.  In  other  words,  knowing  which  of 
the  four  situations  actually  holds  true  is  worth  $0.75  billion  to 
our  decision  maker,  and  if  there  is  any  way  he  can  get  this  information 
at  a  cost  of  less  than  $0.75  billion,  it  will  be  rational  for  him  to 

do  so.  To  be  more  specific,  if  he  can  determine  the  costs  of  the  two 

alternatives  by  carrying  both  into  development,  he  can  afford  to  spend 
up  to  $750  million  on  one  of  the  missiles,  even  though  it  is  eventually 
cancelled. 

In  the  real  world  the  problems  faced  in  making  development  decisions 
are  clearly  much  more  complex  than  in  this  simple  example.  But,  this 
particular  example  shows  why,  in  making  development  decisions,  it  is 
necessary  to  take  into  account  more  than  a  single  point  estimate  of 
how  much  particular  systems  will  cost.  A  thorough  analysis  of 
alternatives  open  to  the  decision  maker,  including  obtaining  more 
information,  requires  an  evaluation  of  the  uncertainty  surrounding  'cost 
estimates  for  alternative  hardware  configurations. 

In  conclusion,  no  one  technique  for  the  evaluation  of  uncertainty  can 
be  recommended  at  this  time.  For  the  most  part,  cost  analysis  studies 
in  the  past  have  either  ignored  uncertainty  or  been  too  qualitatively 
oriented  ,in  their  uncertainty  evaluations.  It  is  recommended  that 
future  cost  analysis  studies  use  and  present  all  quantitative  techniques 
which  the  analyst  can  justify,  however  judgmental  they  may  be.  Thus, 
in  the  future,  there  will  be  a  basis  for  evaluating  the  relative  merits 
of  the  various  techniques  which  is  not  now  available.  The  problem  of 
uncertainty  will  always  be  present  in  estimating  the  costs  of  future 
weapon  systems.  However,  attempts  to  reduce  it  and  to  find  the  best 
methods  for  evaluating  it  must  be  made. 


SENSITIVITY  ANALYSIS 


Sensitivity  analysis  determines  the  significance  of  a  given  variable 
within  some  prescribed  range  of  interests,  and  it  attempts  to  answer  the 
following  questions:  How  does  the  cost  of  a  system  vary  as  a  result 
of  changes  in  the  configuration  of  the  system,  changes  in  its  deploy¬ 
ment  plan,  changes  in  operational  use,  and  so  on?  To  what  elements 
(considering  operational  assumptions  as  well  as  hardware  specifications) 
is  the  total  cost  of  the  system  especially  sensitive  or  insensitive? 

All  cost  analyses  should  have  sections  in  which  the  quantitative  results 
presented  are  open  to  question.  Sensitivity  analysis  identifies  and 
varies  such  factors  in  magnitude  to  analyze  how  the  final  study  results 
are  influenced  by  possible  variability  in  the  questionable  factors. 

It  changes  each  of  the  study  parameters  while  the  others  are  held 
constant  t<?;  analyze  how  a  small  change  in  the  one  parameter  can  affect 
the  total  cost.  Such  an  analysis  can  be  very  valuable  and  can  be 
a  useful  technique  for  dealing  with  some  of  the  problems  of  uncertainty- 
the  more  so  because  statistical  methods  for  deriving  confidence  limits 
and  other  criteria  of  uncertainty  cannot  be  applied  generally  to  cost 
estimates.  In  studying  proposed  future  systems,  numerous  uncertainties 
must  be  recognized  together  with  their  impact  on  system  costs. 

Studies  of  system  historical  data  have  shown  that  perhaps  the  most 
important  reason  for  differences  between  early  estimates  and  final 
costs  is  that  the  configuration  of  the  system  ultimately  obtained 
differs  considerably  from  that  envisaged  early  in  the  program.  Cost 
sensitivity  analysis  deals  explicitly  with  cost  differences  related 
to  differences  in  system  configuration,  and  it  can  therefore  provide 
a  range  of  system  costs  which  is  likely  to  be  a  more  realistic  guide 
than  a  single,  most  probable  cost.  Sensitivity  analysis  highlights 
potential  problem  areas  and  produces  a  concentration  on  the  most 
important  cost  factors.  The  areas  of  greatest  uncertainty  are  found 
and  where  these  are  costly  areas,  options  are  readily  identified  for 
future  research  and  experimentation. 

An  example  of  sensitivity  analysis  is  a  study  done  by  the  US  Army 
Missile  Command  Cost  Analysis  Group.  Congress  was  questioning  the 
possibility  of  using  the  SHILLELAGH  missile,  an  anti-tank  missile 
designed  to  be  fired  by  a  tank,  in  a  ground  mode  rather  than  the  TOW 
missile  which  was  designed  for  ground  operation.  The  costs  of  the 
two  alternatives  were  to  be  considered  with  the  most  inexpensive  one 
being  selected.  The  SHILLELAGH  missile  had  been  produced  for  five 
years  longer  than  the  TOW  and  had  undergone  a  price  decrease  due  to 
competition  among  possible  manufacturers.  It  was  calculated  that 
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competition  effected  a  step  down  in  the  learning  curve  of  about  29%. 
TOW  also  was  planned  to  be  made  competitive  and  therefore  based  on 
SHILLELAGH  experience,  a  29%  factor  was  used.  However,  it  was 
expected  that  this  assumption  was  very  likely  to  be  questioned,  so 
a  cost  sensitivity  analysis  was  conducted.  The  competitive  cost 
reduction  factor  had  no  probability  distribution  associated  with  it, 
so  costs  were  computed  for  factors  of  0%,  10%,  20%,  29%,  and  35%. 

Even  the  0%  competition,  the  total  TOW  program  cost  less  than  the 
total  SHILLELAGH  program  and  an  unequivocal  decision  could  then  be 
made. 

vigure  4  shows  sensitivities  and  insensitivities  revealed  by  costing 
various  configurations  of  a  ballistic  missile  system.* 

1.  The  system  cost  is  relatively  insensitive  to  increases  in 
payload;  that  is,  a  considerable  increase  in  payload  can  be  obtained 
for  relatively  small  increases  in  cost.  The  explanation  is  that  a 
number  of  the  expensive  components  of  the  system  change  little  with 
increases  in  gross  weight  of  the  missile  (e.g.,  guidance  and  control 
systems,  ground  support  equipment,  and  installations  items  associated 
with  fire  control  and  flight  vehicle  guidance), 

2.  The  system  cost  is  relatively  sensitive  to  type  of  propellant 
used.  The  explanation  is  that  the  use  of  solid  or  storable  'liquid 
propellants  eliminates  the  need  for  the  expensive  storage  and  transfer 
facilities  and  equipment  required  by  cryogenic  propellants. 

3.  The  system  cost  is  relatively  sensitive  to  the  automation 
of  the  ground  environment.  The  explanation  in  this  case  is  that  an 
automated  environment  requires  less  launch  site  checkout  equipment 
and  personnel  and  personnel  facilities. 

Figure  5  illustrates  a  boost-glide  system*  which  is  initially  quite 
sensitive  to  increases  in  the  weight  of  the  warhead.  The  cost  curve 
shows  that  the  sensitivity  decreases  with  increasing  warhead  weight, 
so  that  after  a  point  substantial  increases  in  weight  could  be  bought 
for  only  moderate  rises  in  system  costs. 

Figure  6  displays  (on  a  slightly  different  scale)  the  major  elements 
of  cost  which  contribute  to  the  total  cost  curve  shown  in  Figure  5. 
From  this  breakdown,  it  is  apparent  that  the  cost  of  the  operational 
flight  vehicles  is  the  most  important  element  in  the  total  cost  curve, 
although  other  elements  also  contribute  to  its  change  of  slope. 

*  Example  from  System  and  Total  Force  Cost  Analysis,  by  David  NovicU, 
The  Rand  Corporation,  April  1961. 
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Figure  5-4 
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Figure  5-5 
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Figure  5-6 


Figure  7  illustrates  a  case  in  which  the  total  cost  of  a  system 
initially  decreases  and  then  increases  as  one  of  the  system 
characteristics  is  altered  toward  higher  values.  The  system  includes 
a  number  of  satellites  in  orbit,  and  as  their  altitude  is  increased, 
the  cost  of  the  system  falls  markedly  at  first,  primarily  because  the 
change  in  altitude  makes  it  possible  for  fewer  satellites  to  do  the 
job.  But  after  a  point  this  saving  is  counterbalanced  by  the  increasing 
costs  of  other  components:  the  larger  boosters,  more  sensitive 
instrumentation,  and  more  powerful  communication  equipment  needed  for 
the  greater  altitudes.  The  point  is  that  the  cost  implications  of  the 
system  are  fully  revealed  only  by  costing  a  wide  range  of  system 
configurations.  Not  only  can  the  slope  of  the  cost  curve  suggest 
possibilities  of  advantageous  tradeoffs  between  cost  and  system 
characteristics;  in  some  cases,  points  of  minimum  cost  can  be  located 
apart  from  points  of  minimum  performance. 

In  conclusion,  some  guidance  for  the  .inclusion  of  sensitivity  analysis 
in  future  cost  analysis  studies  can  be  presented.  Each  cost  analysis 
should  include  a  separate  section  on  sensitivity  analysis.  It  should 
isolate  the  sensitive  variables  and  present  the  effects  on  cost  due  to 
variations  in  the  value  of  the  sensitive  variables.  The  selection  of 
which  areas  to  study  for  sensitivity  is  generally  obvious  to  experienced 
analysts  associated  with  a  system.  The  analyst  must  also  be  responsive 
to  requests  by  decision-makers  to  vary  the.  assumptions.  In  general, 
the  effects  of  changes  in  buy  quantities  as  well  as  variations  in  a 
minimum  of  three  other  sensitive  parameters  should  be  used  in  the 
sensitivity  analysis  at  this  point  in  time.  As  sophisticated  data 
processing  becomes  more  available,  the  full  range  of  the  element 
estimated  should  be  explored.  For  ease  of  use,  the  results  of  the 
sensitivity  analysis  should  be  summarized  and  presented  in  table  or 
graph  form. 
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CHAPTER'  6 


ECONOMIC  CONSIDERATIONS 


Economic  analysis  and  cost  estimating. 

Economic  analysis  concerns  the  evaluation  of  alternatives.  It  is 
clear  t'  it  any  attempt  to  evaluate  alternative  proposals  or  courses 
of  action  in  terms  of  objectives,  costs,  and  benefits  must  include  a 
consideration  of  cost  information.  Hence,  cost  estimates  play  an 
integral  role  in  any  economic  analysis.  This  chapter  covers  certain 
economic  considerations  that  may  affect  cost  estimates  rather  than  the 
general  subject  of  economic  analysis.  The  concepts  and  procedures  to 
be  followed  in  conducting  economic  analyses  are  described  in  Army 
Regulation  37-13,  "Economic  Analysis  of  Proposed  Army  Investments". 

Time-phasing. 

Most  cost  estimates  must  be  time-phased.  Not  only  is  it  necessary  to 
know  how  much  a  proposed  system  will  cost,  but  it  is  also  necessary  to 
know  when  the  expenditures  will  be  made.  There  are  several  reasons 
for  this.  First,  planning  costs  must  ultimately  be  programmed  into  a 
budget.  Second,  time  phased  cost  estimates  are  required  in  order  to 
make  trade-offs  among  projects  which  have  differing  expenditure  patterns 
over  time.  Third,  if  inflation  is  to  be  considered,  the  distribution 
of  expenditures  over  time  must  be  known. 

Inflation. 

Definitions: 

There  has  been  considerable  confusion  over  the  terms  "constant"  and 
"current"  dollars.  The  following  definitions  are  in  accordance  with 
generally  accepted  usage: 

Constant  Dollars.  The  phrase  "constant  dollars"  is  always 
associated  with  a  base  year  (e.g.,  FY  71  constant  dollars)  even  though 
the  base  year  may  not  always  be  specifically  stated.  An  estimate  is 
said  to  be  in  constant  dollars  if  costs  for  all  work  are  adjusted  so 
that  they  reflect  the  level  of  prices  of  the  base  year.  A  constant 
dollar  estimate  is  what  would  be  paid  if  price  levels  had  always  been 
the  same  and  always  remained  the  same  as  in  the  base  year. 
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Current  Year  Dollars.  Current  Year  Dollars  are  current  to  the  year 
the  work  is  performed.  When  prior  costs  are  stated  in  current  year 
dollars,  the  figures  given  are  the  actual  amounts  paid  out.  When 
future  costs  are  stated  in  current  year  dollars,  the  figures  given 
are  the  actual  amounts  which  will  be  paid  including  any  amount  due  to 
future  price  changes.  The 'turrent  year”  in  "current  year  dollars" 
does  not  refer  to  the  year  in  which  the  estimate  is  made  or  any  other 
single  year. 

The  use  of  constant  dollars. 


The  use  of  constant  dollars  in  budget  requests  has  two  major  benefits. 
First,  constant  dollars  may  be  useful  in  the  consideration  of 
resource  requirements  over  time.  Here,  the  use  of  constant  dollars 
removes  distortions  which  are  attributable  only  to  price  level 
changes.  Second,  using  constant  dollars  aids  in  the  attempt  to  control 
inflation  since  the  expectation  that  inflation  will  continue  adds 
greatly  to  inflationary  pressure. 

The  use  of  current  dollars. 

When  inflation  occurs  between  the  time  a  budget  request  is  submitted 
and  the  time  funds  are  actually  expended,  there  will  be  a  gap  between 
how  far  the  funds  were  supposed  to  go  and  how  far  they  actually  go. 
Hence,  the  use  of  current  dollars  also  has  certain  advantages  for  use 
in  budget  and  programming  documents.  A  major  advantage  is  in 
developing  estimates  which  more  realistically  reflect  likely  expendi¬ 
ture  levels.  Since  cost  estimates  have  been  proven  overly  optimistic 
in  the  past,  inflated  estimates  can  serve  to  reduce  overruns  by 
showing  a  more  realistic  initial  estimate. 

Policy  on  inflation. 

Only  limited  policy  guidance  on  inflation  has  been  announced.  0MB 
is  firmly  adhering  to  its  requirements  for  constant  dollars.  DOD, 
however,  is  making  increasing  use  of  current  dollars  in  its  internal 
planning  process.  Several  items  of  policy  have  been  clearly  defined 
and  should  be  closely  followed.  The  broad  form  of  this  policy  is 
summarized  below.  More  detailed  instructions  are  included  as  annexes. 

General  instructions  have  been  issued  for  Inclusion  of  price 
escalation  in  certain  cost  estimates  for  research  and  development, 
procurement,  and  construction.  This  detailed  guidance  is  inclosed 
as  annex  1. 

Specific  guidance  has  been  issued  for  use  in  the  preparation  of 
Selected  Acquisition  Reports  (SARfs).  This  guidance  is  provided 
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in  annex  2  and  should  be  used  for  SAR  preparation  and  only  for 
SAR  preparation. 

Cost  data  in  constant  dollars  must  be  maintained  in  all  cases.  Since 
the  costs  of  all  systems  must  ultimately  be  programmed  into  the  budget 
cycle,  constant  dollar  costs  must  be  readily  identifiable. 

The  use  of  current  dollars  in  cost  estimates  is  encouraged  whenever 
it  will  aid  in  planning  and  programming  or  whenever  it  may  serve  as 
an  effective  form  of  sensitivity  analysis  (subject  to  the  limiations 
expressed  in  the  detailed  guidance  in  annexes  1  and  2). 

Care  should  be  exercised  to  avoid  double  counting  when  estimating 
unbudgeted  price  increases.  For  example,  if  part  of  a  system  involves 
expenditures  under  a  fixed  price  contract,  these  costs  should  not  be 
Inflated. 

Present  Value. 

Economic  analysis  is  applied  to  alternative  courses  of  action.  The 
emphasis  is  on  comparison  in  a  decision-making  environment.  This  broad 
concept  of  economic  analysis  means  that  the  resultant  decision  is  to 
take  into  account  the  relevant  benefits  and  costs  of  all  appropriate 
alternatives.  Moreover,  the  decision  maker  is  assumed  to  maximize 
efficiency  in  terms  of  these  two  empirically  oriented  concepts.  The 
cost  side  in  any  economic  analysis  blends  in  with,  and  is  a  part  of, 
any  typical  costing  exercise  and  has  been  treated  in  other  parts  of 
the  handbook  under  sections  dealing  with  "estimating  techniques"  and 
the  construction  of  LCCE's.  The  problem  is  usually  presented  to  the 
cost  analyst  in  a  rough  and  general  form  with  the  requirement  to 
conduct  an  economic  analysis.  Within  this  general  framework,  the 
analyst  has  the  option  of  structuring  the  alternative  courses  of 
action.  Relevant  costs  and  benefits  must  then  be  calculated  and 
estimated. 

Economic  analysis  expands  cost  analysis  activities  by  examining  the 
effects  of  the  time-value  of  money  on  the  investment  decision. 

Once  cost/benefit  estimates  have  been  generated,  they  must  be 
time-phased  to  allow  for  the  consideration  of  alternative  expenditure 
patterns.  The  time-value  of  money  is  appropriately  considered  by 
computing  present  value  costs.  Present  value  costs  are  to  be  computed 
by  applying  the  appropriate  10  percent  discount  factors  to  the  time-  * 

phased  expenditure  amounts.  The  Present  Value  costs  are  then  obtained 
as  the  sum  total  of  the  discounted  costs. 
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The  present  value  of  the  alternative  is  the  money  cost  which  would  be 
required  to  finance  the  alternative  —  when  10  percent  could  be  earned 
by  investing  the  funds  until  required  for  expenditure.  If  10  percent 
interest  can  be  earned,  then  this  represents  the  "opportunity  cost" 
of  capital.  Assuming  equal  benefits,  the  alternative  whose  present 
value  cost  is  least  is  the  more  desirable,  because  it  implies  a  more 
efficient  allocation  of  resources.  The  lower  present  value  cost 
means  that  resources  are  allocated  more  efficiently  in  the  sense 
that  fewer  current  resources  must  be  diverted  to  satisfy  the  require¬ 
ment.  The  particular  approach  to  any  economic  analysis  (i.e.,  equal 
cost;  equal  benefit;  unequal  cost;  unequal  benefit)  will  be  dependent 
upon  the  specific  nature  of  the  problem.  No  one  approach  is 
inherently  superior  and  all  are  logically  equivalent. 
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ANNEX  1 


GENERAL  INSTRUCTIONS  FOR  INCLUSION  OF  PRICE 
ESCALATION  IN  CERTAIN  COST  ESTIMATES  FOR  RESEARCH 
AND  DEVELOPMENT,  PROCUREMENT,  AND  CONSTRUCTION 


1.  References: 

a.  DODI  7045.10,  "Five  Year  Defense  Program  Procurement  Annex," 

14  April  1970. 

b.  DOD  Manual  7110. 1-M,  "Manual  for  Preparation  of  Budget  Estimates, 
Operating  Budgets,  Financial  Plans  and  Apportionment  Requests,  and 
Related  Support  Material,"  dated  23  August  1968,  as  changed  13  August 
1970. 


c.  AR  37-100,  "Army  Management  Structure." 

d.  ASD(C)  Memorandum,  30  June  1970,  Subj:  Weapon  System  Costing. 
2.  Applicability. 

a.  This  instruction  applies  to  cost  estimates  appearing  in  the 
Five  Year  Defense  Program  (FYDP),  the  budget,  Development  Concept 
Papers  (DCP),  Selected  Acquisition  Reports  (SAR),  the  Program 
Objectives  Memorandum  (POM),  and  related  items.  Specific  implementing 
instructions  for  these  documents  will  be  provided  as  appropriate. 

b.  This  instruction  applies  to  the  following  costs  with'  the 
exception  noted  in  2b(5). 

(1)  Procurement  costs  for  all  systems  (line  items)  listed 
specifically  by  name  in  the  FYDP  Procurement  Annex  in  Activities  6, 

7,  and  8  (Tactical  and  Support  Vehicles,  Communication  and  Electronics 
Equipment,  and  Other  Support  Equipment)  if  the  procurement  cost 
exceeds  $5  million  in  any  year  without  the  inclusion  of  anticipated 
price  changes  and  in  Activities  1,  3,  and  5  (Aircraft,  Missiles,  and 
Tracked  Combat  Vehicles)  regardless  of  cost.  The  FYDP  Procurement 
Annex  is  prepared  in  the  same  manner  as  the  PEMA  Line  Item  List 
(Exhibit  P-1)  but  covers  the  same  time  period  as  the  FYDP.  Procure¬ 
ment  cost  is  as  defined  in  reference  lb.  This  is  the  same  procurement 
cost  as  is  shown  in  the  SAR  and  is  the  sum  of  the  individual  system 
line  and  its  associated  initial  spares  in  the  FYDP  Procurement  Annex 
and  in  the  P-1.  This  instruction  does  not  apply  to  modifications, 
component  improvement,  replenishment  spares  and  items  listed  in  the 
aggregate  (except  initial  spares  identified  with  systems  to  which 
this  instruction  does  apply). 


) 
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(2)  RDTE  costs  for  systems  reported  under  the  SAR.  RDTE  costs 
for  other  systems  listed  by  name  as  program  elements  or  major 
projects  within  program  elements  in  the  FYDP  will  be  included  in 
the  future  as  instructions  from  HQ  DA  may  direct. 

(3)  Costs  of  Major  Construction  (Budget  Program  6100.0000  and 
Budget  Accounts  6800.2000  and  6800.3000  within  Military  Construction, 
Army,  Budget  Accounts  8511.0000  and  8512.0000  within  Military 
Construction,  Army  National  Guard,  and  Budget  Account  8612.0000 
within  Military  Construction,  Army  Reserve). 

(4)  Costs  of  Construction  of  New  Family  Housing  Units  (Budget 
Project  1810.0000  within  Family  Housing  Management  Account,  Defense  - 
Transfer  to  Army). 

(5)  As  an  exception  to  2b(l)  through  2b(4)  costs  of  compensation 
of  government  employees  are  not  intended  to  be  covered  by  this 
instruction  even  though  they  may  be  included  in  the  above  four 
categories. 

3.  Types  of  Price  Escalation.  A  distinction  is  drawn  between 
the  following  two  types  of  anticipated  price  changes: 

a.  Specific  data:  Specific  data  is  applicable  to  a  given 
system  or  project  and  consists  of  contract  provisions,  labor  agree¬ 
ments,  productivity,  and  quantity  changes  and  considers  the  extent 
to  which  materiel  is  on  hand  or  under  fixed  price  contract.  This 
data  is  based  on  events  which  have  occurred  or  will  definitely  occur 
rather  than  a  general  expectation  or  extrapolation  of  changes.  Specific 
data  is  the  type  of  data  which  has  been  used  as  the  basis  for  including 
anticipated  price  changes  in  prior  budget  requests. 

b.  Price  Level  Indices:  A  price  level  index  forecasts  the 
general  level  of  prices.  The  following  indices  have  been  provided 
by  the  Assistant  Secretary  of  Defense  (Comptroller): 

Family  Housing 


Procurement 

RDTE 

And  Construction 

FY  1970 

96.6 

95.1 

94.0 

FY  1971 

100.0 

100.0 

100.0 

FY  1972 

102.6 

103.8 

104.8 

FY  1973 

104.5 

107.5 

109.2 

FY  1974 

106.1 

110.8 

113.3 

FY  1975 

107.5 

114.1 

117.3 

FY  1976 

108.9 

117.5 

121.4 

After  FY  1976, 

add 

the  indicated 

annual 

increase,  compounding  1.37. 

3.07. 

3.57. 

The  FY  1971  index  should  be  considered  as  the  price  level  prevailing 
in  about  the  fourth  quarter  of  Calendar  Year  1970,  or  the  average 
price  level  prevailing  in  FY  1971. 

4.  Policy.  In  the  documents  to  which  thi9  instruction  is  applicable, 
estimates  of  cost9  specified  in  paragraph  2b  will  reflect  the  best 
estimate  of  the  amounts  ultimately  to  be  paid,  specifically  incorporating 
anticipated  changes  in  future  prices.  Wherever  practicable,  this  will 

be  accomplished  on  the  basis  of  specific  data  applicable  to  a  given 
system  as  defined  in  paragraph  3a.  In  other  cases,  it  will  be  necessary 
to  base  the  estimates  on  forecasts  of  changes  in  price  levels.  In 
such  cases  the  indices  given  in  paragraph  3b,  and  only  these  indices, 
will  be  used. 

5.  Time  Phasing  of  TOA  Expenditure. 

a.  In  projecting  prices,  it  should  be  noted  that  the  TOA 
reflected  for  a  particular  year  must  reflect  the  price  escalation 
over  the  span  of  years  during  which  that  TOA  will  be  expended.  For 
example,  FY  1972  TOA  might  reflect  escalation  from  FY  1972  through 

FY  1975,  taking  account  of  the  time  phasing  of  outlays  on  firm  contracts 
for  phe  system  involved. 

b.  When  price  escalation  is  based  on  indices,  the  indices  should 
ideally  be  applied  to  dollar  amounts  assigned  to  years  on  an  accrued 
expense  basis.  To  apply  indices  to  a  given  year's  TOA,  that  year's 
TOA  should  be  time-phased  on  (or  near)  an  accrued  expense  basis  and 
indices  should  be  applied  against  this  time-phasing  to  arrive  at  the 
escalated  TOA  for  the  original  year.  Inclosure  1  provides  standard 
factors  by  appropriation  for  the  time-phasing  of  expenditures  of  a 
given  year's  TOA.  These  factors  should  be  employed  unless  there  is 
specific  information  available  supporting  a  different  expenditure 
pattern.  Inclosure  2  provides  an  example  applying  indices  to  $1 
million  of  FY  72  PEMA  TOA  with  expenditures  time-phased  by  the  factors 
shown  in  Inclosure  1.  This  example  assumes  that  the  basic  estimate  to 
which  indices  are  applied  is  in  FY  71  constant  dollars;  existing 
estimates  not  in  FY  71  constant  dollars  would  have  to  be  adjusted  to 
FY  71  constant  dollars  before  the  procedure  shown  in  the  example  could 
be  employed.  Since  the  amount  of  price  escalation  to  be  added  to  each 
year's  TOA  is  sensitive  to  the  time  phasing  of  the  expenditures  of 
that  year's  TOA,  offices  preparing  estimates  should  maintain  the 
capability  to  reproduce  the  time  phasing  of  each  year's  TOA  expenditure 
used  in  applying  indices. 
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6.  Double  Inclusion  of  Price  Escalation. 


I 


a.  For  each  system  or  project,  anticipated  price  changes  based  on 
specific  data  should  be  included  in  costs  for  which  they  are  available 
and  price  escalation  based  on  indices  should  be  included  in  those 
costs  for  which  specific  data  on  anticipated  price  changes  is  not 
available. 

b.  In  cases  \  ere  price  escalation  to  cover  several  years  is  to 
be  included,  it  ma  be  appropriate  to  base  price  changes  in  the  near 
future  on  specific  data  and  employ  indices  to  cover  the  remaining  time 
period.  In  general,  it  is  expected  that  anticipated  price  changes  for 
the  budget  year  would  be  based  on  specific  data  while  price  indices 
would  be  used  for  years  farther  beyond  the  budget  year. 

c.  To  avoid  double  counting,  care  must  be  taken  that  a  single 
cost  within  the  estimate  for  a  given  system  or  project  is  not  increased 
once  because  of  specific  data  and  then  again  because  of  indices 
covering  the  same  time  period  as  the  specific  data. 

7.  Visibility  of  Price  Escalation. 

a.  In  order  to  maintain  comparability  with  costs  not  including 
anticipated  price  changes,  it  is  desirable  that  the  amount  of  price 
escalation  included  within  an  estimate  and  the  basis  for  the  price 
escalation  be  stated  either  along  with  the  estimate  or  in  appropriate 
backup. 


b.  Since  the  practice  of  escalation  by  indices  is  subject  to 
errors  having  large  effects  on  costs,  records  will  be  maintained  of 
the  amount  of  price  escalation  included  in  estimates  that  is  based 
on  indices.  Particular  care  in  recording  price  escalation  based  on 
indices  is  required  in  systems  or  projects  where  both  specific  data 
and  price  indices  are  employed. 

8.  Other  Cost  Estimates.  Estimates  of  costs  not  covered  by  this 
instruction  will  be  prepared  as  they  have  been  in  the  past  and  as 
other  instructions  may  provide. 


( 
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ESTIMATED  EXPENDITURE  RATES  FOR 
A  GIVEN  YEAR'S  TOA  (AS  A 
PERCENT  OF  THAT  YEAR'S  TOA) 


APPROPRIATION 

IN 

FIRST 

YEAR 

IN 

SECOND 

YEAR 

IN 

THIRD 

YEAR 

IN 

FOURTH 

YEAR 

IN 

FIFTH 

YEAR 

PEMA  (Other  than  Ammo) 

16.00 

44.00 

28.00 

9.00 

3.00 

RDTE,  Army 

51.00 

38.00 

7.50 

2.10 

1.40 

MCA 

10.00 

50.00 

25.00 

10.00 

5.00 

MCA,  R 

5.00 

51.00 

29.00 

8.00 

7.00 

MCA,  NG 

1.00 

34.00 

45.00 

14.40 

5.60 

Inclosure  1 
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CALCULATION  OF  PRICE  ESCALATION  BY  INDICES  -  TIME  PHASING  OF  TOA 


FY  72  PEMA  TOA  Without  Price  Escalation 
(FY  71  Constant  Dollars)  =  $1,000,000 


FISCAL 

YEAR 

PRICE 

INDEX 

EXPENDITURES  OF  FY  72  TOA  ON  AN  ACCRUAL  BASIS 
(AS  APPROXIMATED  BY  PROJECTED  DISBURSEMENTS) 

In  FY  71  Constant  Dollars  In  Current  Year  Dollars 

(i.e.,  without  price 
escalation) 

(i.e. ,  with  price 
escalation) 

FY  72 

102.6 

160,000 

164,160 

FY  73 

104.5 

440,000 

459,800 

FY  74 

106.1 

280,000 

297,080 

FY  75 

107.5 

90,000 

96,750 

FY  76 

108.9 

30,000 

32,670 

TOTAL 

1,000,000 

1,050,460 

FY  72  TOA  PEMA  with  price  escalation  (Current  Year  Dollars)  =  $1,050,460 


Inclosure  2 
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ANNEX  2 


INSTRUCTIONS  FOR  CALCULATIONS  OF  PRICE  ESCALATION  IN  THE 
SELECTED  ACQUISITION  REPORT  (SAR) 


1.  References: 

a.  General  Instructions  for  Inclusion  of  Price  Escalation  in 
Certain  Cost  Estimates  for  Research  and  Development,  Procurement, 
and  Construction,  dated  4  September  1970,  COMPT-CA(R). 

b.  Composite  Indices  for  Simultaneous  Application  of  Standard  TOA 
Expenditure  Rates  and  General  Price  Level  Indices,  dated  15  September 
1970,  COMPT-CA(R)  (Inclosure  4). 

2.  General:  The  provisions  of  reference  la  will  be  followed  in 
preparing  estimates  for  Selected  Acquisition  Reports.  The  use  of  Price 
Escalation  Worksheets  and  of  a  Price  Escalation  Summary  for  the  SAR 
are  described  below.  These  instructions  do  not  alter  the  formats  used 
in  the  SAR  proper  as  it  is  forwarded  to  OSD  and  Congress.  Additional 
lines  are  required  in  the  Fiscal  Year  Display  to  show  the  amount  of 
future  price  escalation  included.  Previous  estimates  in  the  SAR  have 
included  price  escalation  on  the  basis  of  various  assumptions.  Amounts 
previously  included  in  SAR  estimates  for  future  price  escalation  will 
be  removed  and  only  future  price  escalation  based  on  these  instructions 
will  be  re incorpora ted  in  the  SAR. 

3.  Applicability:  The  types  of  cost  to  which  these  instructions  apply 
are  defined  in  paragraph  2b  of  reference  la.  It  should  be  noted  that 
these  instructions  do  not  apply  to  Additional  Procurement  Costs.  The 
provisions  of  reference  la  will  be  applied  in  the  SAR  to  the  Current 
Estimate,  the  FYDP  Estimate  and  under  Contractor  Costs,  to  the  Govern¬ 
ment  Estimate  of  Price  at  Completion.  The  Planning  Estimate,  Development 
Estimate,  and  Contractor  Costs  (other  than  the  Government  Estimate  of 
Price  at  Completion)  are  matters  of  record  and  cannot  be  changed  by 

the  Project  Manager  because  of  the  new  policy  regarding  anticipated 
price  changes.  Headquarters  DA  will  furnish  the  FYDP  Program  and 
reflect  as  a  separate  line  entry  the  price  escalation  to  be  shown  as 
a  memo  entry.  The  same  principles  that  are  applied  to  price 
escalation  for  the  FYDP  estimate  and  the  current  estimate  will  be  used 
to  compute  the  price  escalation  for  the  FYDP  program. 

4.  Price  Escalation  Worksheet  (Incl  1): 

a.  The  Price  Escalation  Worksheets  are  for  the  Project  Manager's 
use  in  preparing  his  estimates.  Although  the  worksheets  need  not  be 
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submitted  along  with  the  SAR,  they  must  be  available  for  submission 
to  Headquarters  DA. in  the  event  that  the  estimates  must  be  revised  or 
the  procedure  for  handling  price  escalation  must  be  reviewed. 

b.  At  the  top  of  the  format  enter  the  name  of  the  Project,  the 
estimate  (Current  Estimate  and  stub  item,  FYDP  Estimate,  or  Government 
Estimate  of  Price  at  Completion),  Appropriation,  and  the  fiscal  year  of 
the  Total  Obligational  Authority  being  estimated  ("FY70  and  prior",  FY71, 
FY72,  FY73,  FY74,  FY75,  FY76,  or  "To  Complete").  Separate  worksheets 
may  be  used  for  each  fiscal  year  within  "FY70  and  prior"  and  "To 
Complete" . 

c.  Under  "Year  of  Expenditure"  enter  as  column  headings  the  fiscal 
years  in  which  expenditures  will  occur  from  the  TOA  of  the  fiscal  year 
being  estimated.  The  projected  expenditures  of  the  TOA  will  be  time 
phased  in  these  columns  in  order  to  apply  indices. 

d.  Portion  not  subject  to  further  escalation.  Since  indices 
will  not  be  applied  to  amounts  appearing  in  this  section,  expenditure 
time  phasing  is  not  required.  Only  sunk  costs,  costs  arising  from 
signed  contracts  which  establish  a  price  which  will  not  change  due  to 
inflation,  and  compensation  of  government  employees  will  be  listed  in 
this  section.  Government  personnel  compensation  will  be  priced  as 
provided  for  in  Budget  Formulation  Directives  (i.e.,  approved  FY71 
pay  rates  will  be  used  for  future  years).  Note  the  effect  of  the 
update  to  FY  71  pay  rates  on  the  line  indicated. 

e.  Specific  Data:  Contracts  with  Escalation  Clauses: 

(1)  In  order  to  predict  the  price  of  the  work  at  completion  i 
covered  by  a  contract  with  an  escalation  clause,  it  is  necessary 

to  use  the  levels  which  will  actually  be  assumed  by  the  price 
indices  on  which  the  contract's  escalation  clauses  are  based.  The 
OSD  indices,  adjusted  as  described  below,  will  be  treated  as  the 
actual  future  values  of  standard  indices  appearing  in  escalation 
clauses.  By  taking  the  contract  price  of  the  work  financed  by  a 
given  year's  TOA  (including  any  earned  incentives  or  portions  of  the 
target-ceiling  price  spread)  and  adding  the  reimbursement  for 
inflation  resulting  from  execution  of  the  escalation  clause,  the 
estimate  in  current  year  dollars  for  the  TOA  expended  on  the  given 
work  is  obtained. 

(2)  The  OSD  indices  will  be  adjusted  to  put  them  in  the  same  base 
year  as  the  standard  indices  appearing  the  the  escalation  clause 
allowing  for  the  latest  historical  value  of  the  standard  indices. 
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Typically,  contract  escalation  clauses  are  based  on  Bureau  of  Labor 
Statistics  (BLS)  indices  for  labor,  materiel,  subcontract  and  so 
forth.  These  indices  are  on  a  calendar  year  rather  than  fiscal  year 
basis,  and  the  latest  historical  annual  averages  for  these  indices  are 
from  CY  69.  Calendar  year  OSD  indices  with  a  base  year  of  CY  69  are 
provided  at  Inclosure  3.  In  order  to  obtain  index  numbers  to  use  in 
determining  the  effect  of  an  escalation  clause,  multiply  the  historical 
CY  69  value  of  the  standard  index  by  the  index  numbers  from  Inclosure  3. 

If  the  CY  73  value  for  a  BLS  index  were  required  for  an  escalation 
clause  dealing  with  PEMA  funds,  the  historical  CY  69  BLS  index  would 
be  multiplied  by  the  CY  73  procurement  index  from  Inclosure  3.  Similarly, 
if  the  CY  74  value  for  a  BLS  index  were  required  for  an  escalation 
clause  dealing  with  RDTE  funds,  the  historical  CY  69  BLS  index  would 
be  multiplied  by  the  CY  74  RDTE  index  from  Inclosure  3. 

(3)  The  Price  Escalation  Worksheet  will  be  filled  out  in  the 
following  manner.  Each  contract  with  an  escalation  clause  will  be 
handled  separately  (as  a,  b,  c,  ...).  For  each  contract,  list  the 
contract  number  and  in  the  total  column  enter  the  price  of  work  at 
completion  including  the  allowance  calculated  for  the  escalation  clause. 
This  allowance  should  be  shown  as  a  memo  entry  "Future  Price  Escalation". 
Projections  and  calculations  required  to  determine  the  effect  of  the 
escalation  clause  should  be  done  on  a  backup  sheet  attached  to  the 
worksheet. 

f.  Portion  subject  to  escalation  by  indices: 

(1)  This  section  of  the  format  will  be  used  for  costs  which  are  not 
covered  elsewhere. 

(2)  In  the  space  provided  for  "Conversion  to  FY  71  Constant 
Dollars",  note  how  the  portion  subject  to  escalation  by  indices  is 
brought  to  FY  71  constant  dollars.  The  update  of  older  estimates  not 
in  FY  71  constant  dollars  should  be  done  by  use  of  the  OSD  indices  in 
paragraph  3b,  reference  la,  unless  the  use  of  historical  data  can  be 
supported  by  documentation.  If  the  OSD  indices  are  used  to  conver  to 
FY  71  constant  dollars,  this  calculation  should  be  done  here. 

(3)  On  the  line  "FY71  Constant  $"  enter  the  cost  in  FY71  constant 
dollars  of  all  work  to  be  accomplished  with  funds  from  the  FY  of  TOA 
which  is  subject  to  escalation  by  indices.  These  costs  must  be  in 
FY71  constant  dollars.  .  The  amount  shown  on  this  line  must  be  time- 
phased  as  to  expenditure  (with  the  exception  noted  in  paragraph  4f(4)). 
Enter  the  price  indices  from  reference  la  on  the  line  labeled  "OSD 
Index  (FY71  =  100)".  Multiply  the  amounts  in  FY71  Constant  Dollars  by 
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these  indices,  enter  the  products  on  the  line  "Subtotal  Current  Year  $" 
and  enter  the  sum  of  the  time-phased  current  year  dollar  amounts  under 
the  Total  Column.  Show  the  difference  between  "FY71  constant  dollars" 
and  "Subtotal  Current  Year  Dollars"  as  a  memo  entry  "Future  Price 
Escalation. " 

(4)  If  the  standard  expenditure  rates  given  in  Inclosure  1  to 
reference  la  are  used  to  time-phase  the  expenditure  of  TOA,  then  the 
specially  developed  composite  indices  described  in  reference  lb 
(Inclosure  4)  can  be  used  (without  calculating  the  time-phasing  of 
TOA)  to  convert  TOA  for  FY71  and  outyears,  expressed  in  FY71  constant 
dollars,  directly  into  current  year  dollar  totals.  Instead  of  entering 
the  time-phasing  of  expenditures  calculated  f ,om  the  standard  expendi¬ 
ture  rates,  enter  the  notation  "standard  e?  .nditure  rates  used",  and 
enter  the  TOA  in  FY71  constant  dollars  in  cne  Total  Column.  Enter 
the  composite  index  number,  chosen  from  reference  lb  for  the  fiscal 
year  of  the  TOA,  in  the  Total  Column  on  the  line  labeled  "OSD  Index 
(FY71  =  100)."  Multiply  the  TOA  in  FY71  constant  dollars  by  the 
composite  index  number  and  enter  this  amount  on  the  line  "Subtotal 
Current  Year  Dollars." 

g.  Totals:  At  the  top  of  the  worksheet,  in  the  space  marked 
"Total"  show  the  sum  of  the  subtotals  from  sections  1,  2,  and  3. 

Show  the  sum  of  the  memo  entries  from  sections  2  and  3  as  "Future 
Price  Escalation." 

5.  Fiscal  Year  Display  (Inclosure  2):  Upon  completion  of  all 
worksheets,  the  dollar  amount  of  future  price  escalation  within  each 
dollar  entry  in  the  Current  Estimate  and  the  FYDP  Estimate  of  the 
Fiscal  Year  Display  will  be  entered  as  indicated  on  the  attached 
sample  Fiscal  Year  Display. 

6.  Price  Escalation  Summary:  Only  the  results  of  the  price  escalation 
computations  will  be  included  in  the  SAR  as  forwarded  to  OSD.  An 
Inclosure  (no  more  than  one  page,  preferably  one  paragraph)  to  the 
Fiscal  Year  Display  will  summarize  the  logic  and  results  of  the 
following  computations  for  the  FYDP  Estimate,  the  Current  Estimate, 
and  the  Government  Estimate  of  Contract  Price  at  Completion: 

a.  The  update  of  the  constant  dollar  base  from  FY  70  constant 
dollars  to  FY  71  constant  dollars. 

b.  The  price  escalation  computations  for  the  FY  72  budget 
covering  the  projected  update  of  the  FY  71  constant  dollar  base  to 
current  year  dollars. 
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c.  The  future  year  cost  increases  for  price  escalation  (the 
program  from  FY  73  through  completion  of  acquisition). 

7.  Variance  Analysis  and  Changes  from  the  Previous  SAR: 

a.  Changes  and  variances  due  to  causes  other  than  implementation 
of  this  instruction  on  price  escalation  will  be  accounted  for  in  the 
30  Sep  70  SAR  on  the  basis  of  the  pricing  policy  followed  in  the 
30  Jun  70  SAR.  In  the  variance  analysis,  price  escalation  amounts 
should  be  included  in  the  cost  growth  category  "Economic  Change." 

The  "Remarks"  column  should  include  subtotals  of  historic  inflation 
(the  inflation  which  has  been  accounted  for  as  a  result  of  updates  of 
the  SAR  constant  dollar  base  through  FY71)  and  projected  price 
escalation  (the  price  escalation  for  the  budget  year  and  future  years). 


PRICE  ESCALATION  WORKSHEET 
($  MILLIONS) 


Inclosure  1 
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OSD  INDICES  ON  CALENDAR  YEAR  BASIS 


Family  Housing 


Research  and 

and 

Procurement 

Development 

Construction 

CY  1969 

100.0 

100.0 

100.0 

CY  1970 

103.5 

105.3 

106.5 

CY  1971 

106.7 

110.0 

112.5 

CY  1972 

109.1 

114.0 

117.5 

CY  1973 

110.9 

117.8 

122.2 

CY  1974 

112.5 

121.4 

126.6 

CY  1975 

114.0 

125.0 

131.1 

The  CY  1969  index  should  be  considered  as  the  price  level  prevailing 
in  about  the  second  quarter  of  Calendar  Year  1969  or  the  average 
price  prevailing  in  CY  1969. 

These  indices  when  adjusted  for  the  historical  CY  1969  value  of 
the  indices  appearing  in  an  escalation  clause  will  be  used  to  predict 
the  effect  of  the  escalation  clause.  This  procedure  is  necessary 
glnce  no  valid  projection  of  standard  indices  (e.g.,  Bureau  of 
Labor  Statistics  Indices)  is  available.  The  OSD  indices  have  been 
specially  developed  to  apply  to  DOD  activities.  The  index  numbers 
to  be  used  in  calculating  the  effect  of  escalation  clauses  will  in 
effect  be  treated  as  "projections"  of  standard  indices,  but  it  must 
be  realized  that  these  "projections"  are  only  a  combination  of  the 
last  historical  year  of  the  standard  index  and  the  OSD  indices  for 
the  future.  Even  though  a  detailed  analysis  of  the  standard  indices 
would  reveal  differences  between  the  most  probable  future  values 
of  standard  indices  and  the  values  "predicted"  with  OSD  indices,  the 
later  will  be  used. 
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COMPT-CA(R) 


15  Sep  1970 


COMPOSITE  INDICES  FOR  SIMULTANEOUS  APPLICATION 
OF  STANDARD  TOA  EXPENDITURE  RATES  AND  GENERAL  PRICE  LEVEL  INDICES 

1.  Reference:  General  Instructions  for  Inclusion  of  Price  Escalation 
in  Certain  Cost  Estimates  for  Research  and  Development,  Procurement, 
and  Construction,  dated  4  September  1970,  COMPT-CA(R). 

2.  Given  an  amount  of  money  in  FY  71  constant  dollars  as  TOA  for  a 
given  year,  the  time  phasing  of  this  amount  by  standard  expenditure  rates 
and  application  of  price  level  indices  (both  provided  in  ref  1) 
represents  a  simple,  but  time  consuming,  arithmetical  problem,  the 
direct  approach  is  to  multiply  the  TOA  by  each  year's  standard 
expenditure  rate  to  arrive  at  expenditures  for  each  year  in  FY  71 
constant  dollars.  Each  year's  expenditure  is  multiplied  by  that  year's 
price  level  index  to  arrive  at  yearly  expenditures  in  current  year 
dollars.  These  yearly  expenditures  are  summed  to  arrive  at  the  required 
TOA  in  current  year  dollars  (i.e.,  with  price  escalation). 

3.  The  general  formula  for  this  calculation  is  as  follows: 

Yi  =  B^X  +  Bi+iA2  x  +  Bi+2^3  x  +  Bi-+-3A4X  +  Bi+4A5X 
Where: 

X  is  TOA  in  FY  71  constant  dollars 

Y^  is  TOA  in  current  year  dollars  with  expenditures  beginning  in 
year  i. 

A^  through  A^  are  the  standard  expenditure  rates  given  in  ref  1. 

through  Bj^,  are  the  OSD  price  level  indices  for  fiscal  years  i 
through  i+4. 

Note  that  the  above  formula  simplifies  to: 

Yi  =  x  (S-jA^  +  Bi+1A2  +  Bi+2A3  +  Bi+3A4  +  Bi+4A5) 

=  X  ci 

Where  Ct  =  +  Bi+iA2  +  B  1+2^3  +  Bi+3A4  +  Bi+4A5 

Thus,  if  the  factor  (called  herein  the  composite  index  for  fiscal 
year  i)  is  available,  the  calculation  is  reduced  to  one  multiplication. 

4.  The  table  on  the  following  page  shows  the  composite  indices  (by 
appropriation)  for  each  fiscal  year.  Multiplying  a  TOA  amount  in  FY  71 
constant  dollars  by  the  composite  index  for  a  given  fiscal  year  and 
appropriation  automatically  raises  that  TOA  amount  to  current  year 
dollars  allowing  for  time  phasing  (by  standard  rates)  of  expenditures 
beginning  in  the  year  of  authorization. 
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COMPOSITE  INDICES  FOR  SIMULTANEOUS  APPLICATION  OF  STANDARD 
EXPENDITURE  RATES  AND  GENERAL  PRICE  LEVEL  INDICES  1/ 


I 


CHAPTER  7 


PRESENTATION  OF  THE  ESTIMATE 


The  presentation,  both  written  and  oral,  of  the  procedures,  findings, 
conclusions,  and  recommendations  of  a  cost  analysis  study  to  the 
decision  maker  and  other  interested  persons  is  as  important  as 
properly  performing  the  study  itself.  This  phase  in  the  completion 
of  a  study  program  can  mean  the  difference  between  having  a  project 
continued  or  dropped,  or  its  findings  accepted  or  rejected. 

Keeping  this  in  mind  the  writer  must  organize  his  thoughts  and 
material.  The  mere  undertaking  of  a  project  will  require  both  inter¬ 
mediate  and  final  reports,  and  all  of  the  same  steps  in  the  systematic 
accomplishment  of  the  project  or  study  are  involved  in  the  writing  of 
the  project  report.  These  steps  are: 

Stating  the  problem.  This  is  undoubtedly  the  crux  to  the  entire 
study  process.  If  the  wrong  problem  is  addressed,  then  time,  money, 
and  purpose  for  the  study  will  be  wasted. 

Defining  the  scope.  Limits  as  to  what  can  be  presented  within  the 
space  of  time  allotted  must  be  recognized. 

Planning  the  Research, 

Collecting  the  Information,  and 

Analyzing  the  Information:  For  the  purposes  of  cost  analyses 
presentation  these  last  three  steps  involve  having  the  report  preparer 
sit  down  with  the  study  working  papers  and  (1)  deciding  what  needs  to 
be  included  in  order  to  show  how  the  study  conclusions  were  reached, 
(2)  culling  this  information  out  of  the  working  papers,  and  (3) 
determining  how  to  best  set  this  information  down  on  paper. 

THE  USERS:  Following  the  accomplishment  of  these  steps  the  writer 
then  needs,  prior  to  commencing  the  actual  writing  task,  to  consider 
the  controls  which  will  be  influencing  the  way  in  which  he  will  put 
words  down  on  paper.  This  control  takes  the  form  of  the  audience--the 
decision-making  office  that  will  read  the  report  and  use  the  material 
it  contains. 

After  determining  who  the  audience  is  to  be,  the  writer  then  has  a 
basis  for  making  decisions  all  along  the  line,  including  decisions  on 
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such  things  as  the  length  of  the  report,  the  format,  the  writing 
style,  and  the  comprehension  level  of  the  reader.  Knowing  the  audience 
also  includes  knowing  how  the  report  will  be  used.  For  cost  analysis 
reports,  the  use  will  almost  exclusively  be  as  an  aid  in  making  decisions. 
Many,  if  not  most,  of  these  reports  will  ultimately  be  used  by  one  or 
more  individuals  within  the  decision-making  office.  Several  questions 
about  these  people  should  be  answered  in  order  to  aid  report  preparation. 
These  questions  include: 

What  are  their  jobs? 

How  familiar  are  they  with  the  subject? 

What  are  they  interested  in? 

What  are  the  key  questions  they  need  answered? 

Do  they  need  to  be  filled  in  on  some  background  information  or 
have  terms  defined? 

The  more  one  knows  about  these  individuals  the  better  one  can  jnake  the 
report. 

Finally,  one  slight  complication  needs  mentioning.  The  report  may 
in  fact  be  used  by  more  than  one  audience--in  other  words  a  primary 
audience  and  a  secondary  audience.  The  primary  audience  is  seldom 
a  problem  but  the  secondary  audience  can  be  elusive.  It  may  be  one 
person  or  a  hundred.  He  may  use  the  report  next  month  or  pull  it  out 
of  a  file  two  years  from  now.  This  being  the  case  it  is  impossible 
to  anticipate  the  entire  range  of  backgrounds  of  this  type  audience. 

The  report  usefulness  to  the  secondary  audience  can  however  be 
maximized  by  the  conscious  use  of  the  four  traits  of  good  writing 
previously  mentioned. 

THE  REPORT:  Having  determined  who  the  audience,  or  if  possible, 
the  audiences  will  be,  the  writer  can  then  prepare  the  report  itself. 

The  preparation  of  the  results  of  cost  analysis  studies  differs  somewhat 
from  other  subjects  in  that  the  majority  of  the  material  may  be 
numbers,  numerical  comparisons,  and  mathematical  relationships.  The 
writer  will  find  that  tables  are  useful  in  presenting  numbers  and 
numerical  comparisons,  while  statements  of  formulas  or  graphs  will 
best  portray  the  mathematical  relationships.  Tables  have  the 
advantage  of  clarity,  comprehension,  explicitness,  and  economy  of 
space.  Care  must  be  taken  however,  that  the  tables  are  properly 
constructed,  complete,  emphasize  the  proper  comparisons,  avoid 
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extraneous  data,  and  are  properly  labeled.  While  on  the  subject  of 
numerical  presentations,  an  additional  point  needs  to  be  made. 

Numerical  data  is  easy  to  check  by  readers  and  nothing  will  damage  the 
credibility  of  a  study  more  than  a  simple  arithmetic  error.  The  reader 
is  likely  to  think,  "if  this  guy  can't  even  add  and  subtract,  how  can 
he  make  a  good  cost  analysis?"  The  study  and  logic  may  otherwise  be 
flawless  but  the  credibility  will  be  decreased. 

In  the  area  of  graphics,  a  frequent  shortcoming  in  cost  analysis  is  that 
the  writer  fails  to  properly  state  why  a  graph  or  chart  is  relevant  or 
how  it  fits  into  his  analysis.  Often  one  simple  sentence  will  provide 
the  necessary  connection. 

Due  to  the  increasing  cost  growth  of  weapon  systems  over  initial 
estimates,  strong  interest  in  several  areas  of  cost  analysis  has 
developed.  Specifically,  Congress,  and  thus  DOD,  wants  to  know  more 
about  the  influences  of  risk  and  technical  uncertainty  upon  cost.  In 
addition  to  the  detailed  analysis  of  these  influences,  sensitivity 
analysis  including  sensitivity  to  possible  changes  in  assumptions  must 
be  included  as  a  major  section  or  sections  in  the  physical  organization 
of  the  report.  Passing  references  to  these  topics  are  no  longer  adequate 
The  full  report  is  comprised  of  two  major  volumes--the  Executive  Summary 
and  the  Detailed  Report.  The  Executive  Summary  should  be  an  abbreviated 
document  restating  the  findings  and  conclusions  and  summarizing  the 
supporting  material.  Particular  attention  should  be  made  in  pointing  up 
any  reservations  connected  with  the  study  parameters  and  assumptions. 

A  general  format  would  look  as  follows: 

EXECUTIVE  SUMMARY 
I.  TABLE  OF  CONTENTS 
II.  SYSTEM  OVERVIEW 

A.  System  Description 

B.  System  Peculiar  Definitions  or  Aspects 

C.  Study  Objectives 

D.  Assumptions 


III. 


FINDINGS  AND  CONCLUSIONS 


IV.  SUPPORTING  SUMMARIES 


A.  Assumptions 

B.  Variance  Analysis 

C.  Uncertainty/Technical  Risk/Sensitivity  Analysis 

A  Detailed  Report  format  aimed  specifically  at  Weapon/Support 
system  studies  but  modifiable  to  force  studies  is  given  below. 

DOCUMENTATION  FORMAT 

I.  TABLE  OF  CONTENTS 

II .  BACKGROUND 

A.  Authority  for  Study 

B.  System  Overview 

1.  System  description 

2.  System  status  in  the  life  cycle  management  plan 

3.  System  peculiar  definitions  or  aspects 

III.  STUDY  OBJECTIVES  AND  ESSENTIAL  ELEMENTS  OF  ANALYSIS  (EEA) 

A.  Objectives 

B.  Essential  Elements  of  Analysis 

IV.  STUDY  PARAMETERS,  ASSUMPTIONS 

A.  Parameters 

B.  Assumptions 

V.  DATA  SUMMARIES 

VI.  VARIANCE  ANALYSIS 

VII.  UNCERTAINTY/TECHNICAL  RISK/SENSITIVITY  ANALYSES 
A.  Cost  Estimating  Uncertainty 
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B.  Technical  Risk  Cost  Impact 

C.  Sensitivity  Cost  Impacts 
VIII.  FINDINGS  AND  CONCLUSIONS 

ANNEXES: 

ANNEX  1 --METHODOLOGY  (RATIONALE) 

ANNEX  2- -DATA  INPUTS  (IN-HOUSE,  OTHER  SOURCES) 

ANNEX  3 --GLOSSARY 
ANNEX4- - BI BLIOGRAP  HY 

A  brief  explanation  of  each  paragraph  will  assist  in  understanding 
the  contents  of  each  paragraph, 

DOCUMENTATION  FORMAT 

I.  TABLE  OF  CONTENTS  -  Self-explanatory. 

II.  BACKGROUND 

A.  Authorization  for  Study 

This  section  contains  the  letter  of  transmittal  and  the 
authorization  or  directive  which  initiated  the  study.  Background 
information  that  may  provide  perspective  should  also  be  included  in 
this  section. 


B.  System  Overview 

1.  System  Description 

This  section  is  intended  to  provide  the  reviewer 
with  a  perspective  of  the  overall  system.  As  a  minimum,  this 
description  includes  the  operational  concepts,  major  hardware  item 
functions  and  description,  system  performance  characteristics,  main¬ 
tenance  concepts,  and,  where  applicable,  mobility  and  air  transportability 
requirements,  and  other  areas  believed  to  be  necessary. 
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2.  System  Status  in  the  Life  Cycle  Management  Plan 

This  section  contains  a  brief  description  of  the  system 
life  cycle  management  plan  with  emphasis  placed  on  the  current  status  of 
the  system  within  the  plan.  The  life  cycle  management  schedule,  the 
milestone  schedule,  and  the  major  decision  points  should  be  addressed  in 
the  above  context. 


3.  System  Peculiar  Definitions  or  Problems 

This  section  lists  and  defines  those  definitions 
or  problems  which  are  peculiar  to  the  system. 

III.  STUDY  OBJECTIVES  AND  ESSENTIAL  ELEMENTS  OF  ANALYSIS  (EEA) 

A.  Study  Objectives 

This  section  of  the  report  will  exhibit  the  reasons  that 
promoted  the  conduct  of  the  study  and  the  study  objectives.  This 
discussion  should  include  a  reference  to  the  status  of  the  system  within 
the  life  cycle  management  plan  and  identification  of  the  decisions  that 
will  be  influenced  by  the  study. 

B.  Essential  Elements  of  Analysis  (EEA) 

This  section  should  list  the  EEA  that  the  study 
investigated.  These  EEA  are  the  major  issues  or  decisions  involving 
system  life  cycle  costs  that  are  directly  related  to  the  study 
objectives.  They  address  the  major  decision  or  ,?what  if”?  questions 
which  can  be  anticipated. 

IV.  STUDY  PARAMETERS,  ASSUMPTIONS 

A.  Study  Parameters 

This  section  contains  the  initial  guidance  for  the 
study.  It  states  the  place  of  the  weapon  system  in  the  DA  plan  and  a 
general  tactical  employment  concept.  It  also  includes  the  basis  of 
issue  contained  in  the  Table  of  Allowances,  Table  of  Organization  and 
Equipment,  Manning  Table  and  Equipment  List,  or  other  similar  authori¬ 
zation  which  provides  the  basis  of  the  study  and  specifies  the  number  of 
items  issued  to  an  individual,  a  unit,  a  military  activity,  or  for  a 
unit  piece  of  equipment.  This  section  includes  the  total  quantity  to 
be  procured,  the  production  rate  and  schedule,  the  build-up  rate  and 
time  period,  and  the  delivery  schedule.  If  more  than  one  total 
production  quantity  is  considered,  then  the  production  rate,  schedule, 
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and  delivery  schedule  for  each  total  quantity  is  specified.  The 
maintenance  plan  and  requirements  are  discussed  to  include  maintenance 
float,  combat  consumption,  mobilization  reserve,  and  basis  of  issue  of 
special  equipment.  The  initial  provisioning  policy  and  factors  broken 
out  between  PEMA  and  stock  fund  are  provided.  Ammunition  requirements 
are  identified  as  to  type,  basic  load,  initial  provisioning  factors, 
interface  with  other  weapon  systems,  training  ammunition  requirements 
for  introductory  training,  annual  training  allowances,  and  quality 
assurance  requirements.  Operating  cost  assumptions  include  the  Table 
of  Organization,  primary  weapon  crew  size,  annual  operating  factors, 
such  as  mileage,  ammunition  expenditure,  time  between  overhaul, 
maintenance  plan,  replacement  training  plan,  annual  training  program, 
and  accession  factors. 

B.  Assumptions 

This  section  should  include  a  complete  listing  of  the 
statement  of  assumptions  and  the  source  or  justification  for  each 
assumption. 


V.  DATA  SUMMARIES 

/  This  section  consists  of  a  narrative  description  of  the  wrok 
breakdown  structure  (WBS)  used,  and  cost  tables  and  formats  prepared  in 
accordance 'with  AR  37-18. 

/ 

VI.  VARIANCE  ANALYSIS 

This  section  contains  all  data  necessary  to  explain 
changes  between  this  cost  estimate  and  previous  cost  estimates,  if 
applicable. 

VII.  UNCERTAINTY/TECHNICAL  RISK/ SENSITIVITY  ANALYSES 
A.  Cost  Estimating  Uncertainty 

This  section  should  be  devoted  to  the  explanation  and 
quantitative  display  of  confidence  ratings  and  levels  pertaining  to  the 
study  cost  estimates.  Uncertainty  may  be  due  to  many  factors 
(resulting  in  both  measured  and  estimated  uncertainty)  and  the  cost 
analyst  must  consider  all  aspects  to  arrive  at  final  measures  of 
confidence.  The  confidence  ratings  may  be  obtained  by  engineering 
judgment  or  statistical  analysis-pref errably  a  combination  of  both. 
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B.  Technical  Risk  Cost  Impacts 

This  section  addresses  the  potential  error  in  cost 
estimates  due  to  the  degree  of  uncertainty  surrounding  selected  technical 
aspects  of  the  program.  Advanced  systems  concepts  usually  are  based  upon 
assumptions  that  certain  state-of-the-art  advancements  will  make 
necessary  devices  or  techniques  available  at  specific  times  or  after 
the  expenditure  of  specific  amounts  of  resources  in  the  system  develop¬ 
ment  process.  Each  such  assumption  introduces  a  risk  that  a  level  of 
performance  capability,  a  cost,  or  a  schedule  cannot  be  attained  as 
planned.  Possible  consequences  are  severe  time  delays,  extreme  cost 
increases,  major  program  redirection,  or,  in  extreme  cases,  program 
termination.  Comprehensive  development  planning  requires  the  identi¬ 
fication  of  high  risk  areas  and  definition  of  alternative  courses  of 
action  that  can  be  pursued  if  the  desired  results  are  not  achieved. 

For  example,  if  a  new  engine  fails  to  perform  as  anticipated,  what 
would  be  the  cost  impact  of  going  to  a  conventional  engine?  The 
cost  estimator  accounts  for  this  technical  risk  by  calculating  the 
estimated  cost  of  the  alternative  courses  of  action. 

C.  Sensitivity  Cost  Impacts 

This  section  addresses  the  impact  of  the  cost 
fluctuations  in  the  program  due  to  a  change  in  a  study  parameter  or 
assumption.  For  example,  what  is  the  cost  impact  of  the  addition  of 
one  battalion  of  equipment?  What  would  be  the  cost  impact  of  a  one- 
year  R&D  slip?  What  would  be  the  total  cost  impact  of  a  change  in 
the  production/ procurement  rate?  In  effect,  what  is  the  impact  on  the 
Life  Cycle  Cost  Estimate  (LCCE)  and  study  results  which  result  from 
changes  in  the  critical  assumptions  which  were  established  as  the  basis 
for  the  estimate? 

VIII.  FINDINGS  AND  CONCLUSIONS 

This  section  will  summarize  and  present  the  cost 
implications  by  category  of  costs.  Particular  attention  will  be 
given  to  abnormalities,  areas  of  concern  (risk  or  sensitivity),  and 
adequacy  of  inputs.  Conclusions  presented  should  address  any 
applicable  statements  concerning  the  study  parameters  and  assumptions 
used.  The  Resource  Management  Corporation  (RMC)  study,  "Guidelines 
for  Presentation  of  Army  Cost  Analyses”  Draft  Report  UR-125,  May  1970 
should  be  used  as  a  guide  to  develop  graphic  charts. 
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ANNEXEX  TO  THE  DOCUMENTATION  FORMAT 


ANNEX  1:  METHODOLOGY 

The  rationale  should  be  included  in  each  of  the  four  life  cycle 
categories  of  R&D,  Investment  (Non-recurring),  Investment  (recurring) 
and  Operating  Costs.  It  should  be  a  detailed  description  of  the 
procedures  (methodology)  used  to  determine  the  element  cost  in  the 
WBS  (parametric,  engineering,  catalogue  price,  etc.).  It  identifies 
the  costing  rationale  used  for  each  WBS  element.  It  specifies  the 
sources  of  all  cost  statistics  and/or  learning  curves  used  in  costing 
each  WBS  element.  A  detailed  explanation  of  hew  learning  curves  and 
first  unit  cost  were  determined  will  be  shown.  Where  a  computer  is 
used,  it  guides  the  reviewer  through  the  program  logic  using  sample 
calculations.  It  provides  the  clear,  logical,  basic,  defensible 
origin  of  all  cost  numbers. 

ANNEX  2:  DATA  INPUTS 

This  section  should  include  copies  of  the  exact  inputs  which  were 
provided  by  the  in-house  directorates  (RDT&E,  PAP,  Maintenance,  Quality 
Assurance,  etc.)  and  other  commodity  commands  for  use  in  the  study. 

If  deviations  were  made  by  the  study  conductors  in  manipulation  of 
the  actual  data  inputs,  the  rationale  for  making  the  changes  should 
be  explained. 

ANNEX  3:  GLOSSARY 

This  section  should  describe  all  major  terms  and  acronyms. 

ANNEX  4:  BIBLIOGRAPHY 

This  section  should  show  all  source  material  by  organization, 
date,  summary  of  data  or  information  obtained,  person  contacted;  his 
name  and  telephone  number. 

THE  REVIEWER: 

Having  gone  directly  from  the  writer  to  the  organization  of  the 
report  he  was  writing,  one  needs  to  return  to  what  is  a  very  important 
step  in  the  production  of  a  cost  analysis  study.  This  is  the  job  of 
the  reviewer.  Before  a  report  reaches  the  publication  phase  it  will 
be  reviewed  by  someone  other  than  the  writer  for  completeness  and 
clarity.  The  writer  can,  therefore,  help  this  process  along  if  he 
knows  what  the  reviewer  will  be  looking  for  (aside  from  the  obvious 


7-9 


miscalculations  and  misspellings).  The  type  of  information  which  will 
generally  be  looked  for  in  cost  studies  is  covered  in  the  Research 
Analysis  Corporation  (RAC)  publication,  "Guide  for  Reviewers  of  Studies 
Containing  Cost-Effectiveness  Analysis"  (AD  618  892).  While  this  is 
aimed,  as  the  title  indicates,  specifically  at  cost  effectiveness  the 
following  list  of  "selected  questions"  taken  from  that  study  is  most 
pertinent.  If  the  writer's  work  provides  positive  answers  to  these 
and  perhaps  other  applicable  questions  posed  in  the  RAC  document  then 
a  minimum  of  rewriting  will  be  required. 

"SELECTED  QUESTIONS" 

Is  the  problem  stated  in  the  real  problem? 

Are  the  assumptions  unduly  restrictive? 

Are  any  feasible  and  significant  alternatives  omitted 
(if  alternatives  are  applicable)? 

Is  the  study  adequately  documented? 

Are  the  facts  stated  correctly? 

Are  the  cost  estimates  relevant? 

Are  incremental  costs  considered? 

Is  an  amortized  (if  applicable)  cost  used? 

Are  the  models  (cost)  adequately  identified  and  explained? 

Are  the  models  or  methodology  intuitively  acceptable? 

Are  all  criteria  used  consistent  with  higher  echelon 
objectives? 

Have  all  the  significant  ramifications  been  considered  in 
arriving  at  the  findings  and  conclusions? 

Are  the  conclusions  intuitively  satisfying? 

THE  ORAL  REPORT 

Oral  briefings  on  cost  analysis  results  are  to  present  information  and 
not  to  sell  a  system.  The  material  presented  forms  a  portion  of  the 
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basis  upon  which  a  decision  can  be  made  and  if  the  system  is  a  good 
one  it  will  sell  itself.  The  briefing  of  study  results  will  also  save 
time.  It  saves  the  decision-maker's  time  in  that  being  well 
constructed  and  organized,  the  main  facts,  implications  and  conclusions 
resulting  from  the  cost  analysis  are  quickly  and  concisely  revealed. 

This  greatly  simplifies  a  difficult  subject  by  culling  out  all 
extraneous  back-up  material  and  verbage. 

BRIEFING  REQUIREMENTS 

Just  as  in  preparing  the  written  report,  the  reason  for  an  oral 
briefing  must  be  analyzed.  We  have  already  accomplished  one  step  in 
this  process  by  defining  the  presentation  of  cost  analysis  results  as 
an  informational  type  of  briefing.  This  begins  to  shape  the  material 
and  data  to  be  included.  To  finalize  the  amount  and  type  material 
needed  the  briefer  must  next  determine  to  whom  the  briefing  will  be 
given.  It  may  be  someone  who  is  one  or  two  steps  higher  in  the  chain 
of  command  or  it  might  be  someone  outside  of  the  organization.  In 
each  case  the  amount  or  type  of  general  information  included  will  be 
a  little  different.  Just  as  in  the  writing  of  the  report  it  should  be 
determined  as  nearly  as  possible  just  what  the  audience  wants  or  needs 
to  know.  If  one  briefs  his  immediate  superior  the  methodology  and  logic 
by  which  he  arrived  at  specific  numbers  might  have  to  be  discussed 
and/or  defended  while  at  higher  echelons  of  management  such  topics 
probably  would  be  inappropriate.  Another  point  to  determine  is  when  the 
briefing  is  to  be  given.  Usually  this  will  be  mentioned  at  the  time 
the  requirement  for  the  briefing  is  assigned.  This  information  will 
define  the  amount  of  time  available  to  prepare  the  material  and  the  type 
of  graphics  that  can  be  produced.  If  the  date  of  the  briefing  is  far 
enough  in  the  future  a  graphics  department  can  put  together  neat, 
professional  Vu-graphs  and  charts.  However,  should  the  briefing  be  that 
afternoon  or  the  next  day  or  so  then  perhaps  rough  charts  are  the  best 
that  can  be  prepared.  This  should  not  concern  the  person  preparing  the 
briefing  since  it  is  the  information  presented  that  is  important.  A 
fourth  item  in  the  analysis  process  is  to  answer  the  question,  "Where  is 
the  briefing  to  be  given?"  The  answer  will  have  a  direct  bearing  on 
the  type  of  graphics  used.  Should  the  appointed  place  of  presentation 
be  other  than  "home  base"  then  perhaps  more  Vu-graphs  will  be  required, 
or  rough  instead  of  finished  charts.  These  can  sometimes  be  made  at  the 
place  the  briefing  is  to  be  given,  thus  reducing  the  amount  of  material 
that  needs  to  be  carried  from  one  place  to  another. 
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After  completing  the  analysis  of  the  requirements  for  the  briefing, 
the  preparer  can  begin  to  go  through  the  study  material  and  select  that 
data  and  only  that  data  which  is  necessary  to  present  a  clear  and 
complete  picture  of  the  study.  Too  much  data  will  either  bore  the 
listener  or  lead  him  into  tangent  subjects  which  may  destroy  the  purpose 
of  the  briefing.  Thus,  what  the  briefer  must  aim  for  is  to  summarize 
the  analysis--hitting  only  the  salient  points.  He  must  direct  and 
control  the  presentation  so  that  questions  concerning  details  can  be 
brought  up  after  the  briefing  itself  is  concluded.  It  must  be  kept  in 
mind  however,  that  the  material  covered  should  contain  enough  information 
so  that  important  details  aren't  hidden. 

Specifically,  cost  analysis  briefings  should  include  as  appropriate: 

PURPOSE  OF  THE  ANALYSIS.  This  is  required  in  each  case. 

Unless  guidelines  are  laid  out,  lack  of  direction  may  lead  to  a 
compilation  of  unorganized  facts. 

DESCRIPTION  OF  THE  SYSTEM.  This  tells  the  audience  what  the 
data  covered  pertains  to  and  how  one  piece  of  information  relates 
to  another. 

STATUS  OF  LIFE-CYCLE  AND  ENVIRONMENT.  The  status  will  inform 
the  listener  what  has  been  accomplished  up  to  that  point  in  time  and 
whether  the  same  "rules"  apply  as  when  the  study  began. 

MORE  DETAILED  PURPOSE  OF  THE  ANALYSIS.  Such  a  statement  or 
explanation  may  or  may  not  be  necessary.  Such  an  additional  require¬ 
ment  could  be  appropriate  if  the  original  study  was  altered  after 
system  investigation  had  already  begun. 

METHODOLOGY  USED  IN  ANALYSIS.  In  keeping  with  the  philosophy  of 
presenting  just  the  salient  facts,  an  explanation  of  the  methodology 
used  should  be  included  only  if  requested  by  the  person  or  persons 
being  briefed  or  if  some  new  or  unusual  technique  was  added  to  existing 
methodology. 

RESULTS  OF  THE  ANALYSIS.  This  is  the  main  body  of  the  briefing. 

Here  the  data  used,  the  inter-relationships  developed  and  the 
findings  and  conclusions  are  stated.  Again,  enough  information  should 
be  given  that  details  are  not  hidden.  However,  too  much  information 
may  detract  from  the  main  study  purpose. 

PROBLEMS,  UNCERTAINTIES  AND  SENSITIVITIES.  Finally,  all  problems, 
uncertainties  and  sensitivities  must  be  identified  and  stated.  Not 
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to  do  so  may  lead  a  decision-maker  (audience)  who  uses  cost  analysis 
results  in  combination  with  other  information  to  reach  an  erroneous 
conclusion. 

The  use  of  graphics  both  in  written  reports  and  in  briefings  allows 
numerical  data  to  be  conveniently  summarized  and  compared.  They  are 
also  useful  in  uncovering  important  facts  that  might  be  overlooked  in 
written  text  or  tabular  presentations  of  data.  To  use  a  chart 
effectively,  three  basic  steps  should  be  followed: 

1.  SELECT  one  specific  type  that  will  be  most  appropriate--one 
which  explains  the  desired  relationships  and  shows  the  trends. 

2.  DESIGN  it  to  focus  attention  on  the  specific  point  the  chart 
is  intended  to  emphasize. 

3.  CONSTRUCT  the  chart  in  such  a  way  that  it  is  easy  to  read  and 
understand.  Caution  must  be  maintained  so  as  to  avoid  putting  too 
much  information  on  the  chart. 

Finally,  the  sequence  of  the  oral  presentation  must  be  in  a  logical 
order  consistent  with  the  study.  The  "big  facts"  of  the  analysis  should 
be  presented  early  to  draw  attention  to  them  quickly.  These  should 
be  reinforced  with  the  lesser  facts  throughout  the  presentation.  To 
wrap  up  the  briefing  the  conclusions  are  then  listed  so  as  to  complete 
the  study  picture.  Thus  successful,  boil  down  to  a  careful  evaluation 
of : 

THE  PURPOSE 

THE  AUDIENCE 

THE  FACTS  TO  BE  PRESENTED 

Careful  analysis  of  each  of  these  three  areas  before  beginning  to  speak 
will  assure  that  the  presentation  is  not  only  a  medium  of  service  to 
the  listeners,  but  to  the  briefer  as  well. 

CONCLUSION 

In  summary: 

1.  The  steps  in  successful  report  writing  proceed  logically  from 
getting  in  a  good  attitude  for  writing,  to  finding  out  all  about  the 
audience(s),  to  finally  writing  the  report  so  as  to  facilitate  the 
purpose(s)  of  this  audience(s). 
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2.  An  executive  summary  volume  to  a  cost  analysis  is  very  helpful 
to  the  decision  maker  by  summarizing  these  topics  and  presenting 
pertinent  facts  and  conclusions  in  a  clear  and  concise  manner.  This 
document  should  be  cross-referenced  with  the  detailed  main  report  in 
order  that  specific  cost  questions  can  be  answered  if  required. 

3.  The  same  three  steps  mentioned  above  also  apply  to  preparation 
of  an  oral  briefing  on  the  study.  The  presentation  should  address  the 
problem  and  strictly  avoid  extraneous  material.  Annex  1  is  a  briefer's 
check  list  used  by  the  US  Army  Management  School  and  is  a  helpful 
detailed  guide.  Conscious  use  of  the  ideas  presented  in  this  chapter, 
and  more  particularly  by  the  entire  handbook,  will  aid  in  the  success 
of  any  cost  analysis  undertaken. 


U.  S.  ARMY  MANAGEMENT 
SCHOOL 


OPERATIONS  RESEARCH/ SYSTEMS  ANALYSIS 
EXECUTIVE  COURSE 


CHECK  LIST  -  BRIEFING  TECHNIQUES 

1.  WHY  BRIEFINGS? 

a.  Save  time;  of  an  executive. 

b.  Simplify;  make  a  difficult  subject  easy. 

c.  Sell;  information  or  decision,  and  in  either  case,  yourself. 

2.  BRIEFING  REQUIREMENTS 

a.  Get  requirement  in  writing.  If  received  verbally,  put  in 
writing  and  check  with  originator. 

b.  Determine,  from  an  analysis  of  the  requirement: 

WHO  gets  the  briefing?  This  sets  the  level  of  your  presentation. 
WHAT  does  he  want  to  know?  Or,  in  your  opinion,  what  does  he 
need  to  know? 

WHEN  is  the  briefing  due?  This  tells  you  how  much  time  is 
available. 

WHERE  is  the  briefing  to  be  given?  G$t  as  favorable  a  location 
as  possible. 

WHY  is  the  briefing  required?  Dig  out  the  "why"  if  not  readily 
apparent. 

HOW  should  the  briefing  be  presented?  Do  it  like  "WHO"  wants 
it.  Use  "rough"  charts  or  vu-graphs.  Vu-graphs  should  be 
used  only  if  time  permits  their  preparation. 

3.  SELECT  DATA.  Use  only  that  necessary  to  your  presentation. 

4.  BOIL  TO  "BONES".  Or,  what  are  the  salient  points? 

5.  CHART  "BONES"  AS  OUTLINE.  In  preparing  your  "roughs" 

a.  Layout  ,-  you  don't  have  to  be  an  artist.  Anyone  can  make 
rough  charts. 

(1)  Make  your  layouts  on  a  GI  tablet  -  Then  check  with  your 
supervisor  before  starting  "rough"  charts. 
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(2)  Use  both  sides  of  ruler  to  draw  lines  for  lettering  on 
your  rough  charts. 

(3)  Letter  each  line  in  pencil  first,  then  use  MAGIC  MARKERS 
or  crayon. 

(4)  Leave  plenty  of  white  space  top,  bottom,  sides,  and  between 
lines. 

b.  Lettering 

(1)  Plain 

(2)  Minimum  height  IV'.  The  chart  must  be  legible  to  everyone 
in  the  room. 

(3)  Maximum  -  2  or  3  colors.  Suggested: 

(a)  Headings  -  RED 

(b)  Text  -  Black,  key  words  underlined  in  red. 

c.  Content 

(1)  Omit  EXTRANEOUS  matter.  If  you  don't  refer  to  it  in  your 
presentation,  it's  extraneous. 

(2)  Present  SIMPLY. 

(3)  Graph  RELATIONSHIPS.  Reduce  tabular  data  to  vectors,  to 
facilitate  detection  of  trends  by  inspection. 

d.  Sequence 

(1)  Logical  Order  -  Normally,  abbreviated  staff  study  sequence 
(problem,  assumptions,  facts,  discussion,  conclusions, 
recommendations;  headings  unnecessary). 

(2)  Hit  "big  sell"  early.  Inspire  confidence  with  "big"  facts 
first. 

(3)  Reinforce  heavily  with  lesser  facts. 

(4)  End  with  Conclusions  and  Recommendations,  as  appropriate. 

6.  REVIEW  PURPOSE  -  Make  sure  your  conclusions  or  recommendations  do 
in  fact  accomplish  your  purpose. 

7.  PRIVATE  REHEARSAL  -  With  your  final  charts,  builds  up  your  confidence. 

8.  REHEARSE  -  With  a  Critical  Audience  -  Better  be  shot  down  by  friends 
privately  than  by  higher  authority,  in  public.  Don't  be  sensitive. 

9.  MAKE  NECESSARY  CHANGES  -  A  chart  is  a  medium  of  communication  not 
a  work  of  art,  and  rough  charts  are  easily  changed,  by  "patching" 
your  mistakes. 

10.  FINAL  PRESENTATION 

a.  Preparation  -  Get  there  early.  Have  all  aids  ready. 

b.  Have  documentation  available  for  ready  reference. 

c.  Introduce  yourself  if  the  audience  does  not  already  know  you, 
and  state  your  purpose. 
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d.  Talk  from  charts  not  to  them.  Use  pointer. 

e.  Oratory  not  needed.  Speak  clearly  in  your'  natural  voice. 
You  MUST,  however  be  audible  to  everyone  in  the  room. 

f.  Be  sincere  -  You  can't  sell  it  if  you  don't  believe  in  it. 

g.  End  with  Cone lusions  and  Recommendations ,  as  appropriate. 

h.  Answers  -  Don't  dodge.  Answer  only  when  you  know  the 
answers.  Otherwise,  you  destroy  confidence. 

11.  SUMMARY 


a. 

b. 

c. 


g> 

h. 

i. 


Briefings:  Save  time,  simplify,  and  sell. 

Obtain  written  requirement. 

Select  only  pertinent  data. 

Boil  data  to  "Bones". 

Chart  "Bones"  as  outline. 

Follow  staff  study  sequence. i 
Load  with  your  ace. 

Rehearse  with  critical  audience. 

Presentation  is  narrative  built  around  charts  which  tell 
your  story. 


12.  CONCLUSION ;  Briefing  SUCCESSFUL  if: 


a.  Informational  -  You  have  made  your  points. 

b.  Decision  -  You  have  the  desired  answer. 

c.  You  deserve  an  accolade  if  everyone  in  audience  could 
give  briefing,  using  your  charts. 
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CHAPTER  8 


COST  GROWTH 


Cost  growth  is  the  net  change,  which  can  be  positive  or  negative, 
of  a  currently  estimated  or  actual  amount  from  a  base  figure  previously 
established.  The  base  must  relate  to  a  program,  project,  or  contract 
and  be  clearly  identified  including  source,  approval  authority,  specific 
items  included,  specific  assumptions  made,  date,  and  amount. 

Deputy  Secretary  of  Defense  Packard  has  defined  cost  growth  and  listed 
nine  categories  that  will  be  used  in  the  Department  of  Defense.  These 
terms  should  be  used  in  all  cost  studies  to  identify  any  cost  growth 
which  may  have  occurred  since  previous  cost  estimates.  Categories  of 
cost  growth  are  as  follows: 

1.  Engineering  Change  -  An  alteration  in  the  physical  or 
functional  characteristics  of  a  system  or  item  delivered,  to  be 
delivered,  or  under  development,  after  establishment  of  such  charac¬ 
teristics. 

2.  Quantity  Change  -  A  change  in  quantity  to  be  procured,  computed 
at  the  original  price  after  making  appropriate  and  consistent  adjust¬ 
ments  for  cost-quantity  relationships  (e.g.,  improvement  curves), 
thereby  excluding  that  portion  of  the  current  price  attributable  to 
changes  in  any  other  category. 

3.  Support  Change  -  A  change  in  support  item  requirements  (e.g., 
spare  parts,  training,  ancillary  equipment,  warranty  provisions, 
Government-furnished  property/ equipment,  testing,  etc.). 

4.  Schedule  Change  -  A  change  in  a  delivery  schedule,  completion 
date,  or  immediate  milestone  of  development  or  production. 

5.  Unpredictable  Change  -  A  change  caused  by  Acts  of  God,  work 
strikes,  Federal  or  State  law  changes,  or  other  similar  unforeseeable 
events.  Unforeseeable  events  include  contractual  actions  without 
consideration  pursuant  to  PL  85-804  for  (1)  essential  contract 
performance  and  (2)  correction  of  mistakes.  Formalization  of  informal 
commitments  under  PL  85-804  are  not  included  under  this  category  but 
should  be  reflected  under  the  other  categories,  as  appropriate. 

6.  Economic  Change  -  A  change  due  to  the  operation  of  one  or  more 
factors  of  the  economy.  This  includes  specific  contract  changes  related 
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to  economic  escalation.  This  also  includes  changing  real  dollar  amounts 
in  program  estimates  to  reflect  (1)  revised  economic  impact  or  (2) 
definitized  contract  amounts. 

7.  Estimating  Change  -  A  change  in  program  or  project  cost  due  to 
refinements  of  the  base  estimate.  These  include  mathematical  or  other 
errors  in  estimating,  changing  the  base  year  of  the  constant  dollars, 
revised  estimating  relationships,  changing  from  constant  dollars  to 
real  dollars,  etc. 

8.  Contract  Performance  Incentives  -  A  net  change  in  contractual 
amount  due  to  the  contractor's  actual  performance  being  different 
than  was  predicted  by  performance  (including  delivery)  incentive 
targets;  as  differentiated  from  cost  incentive  targets;  established  in 
a  EPI  or  GPIF  contract.  This  category  also  includes  any  changes  in 
amounts  paid  or  to  be  paid  a  contractor  due  to  (1)  award  fee  for 
performance  accomplishments  under  a  cost  plus  award  fee  contract  or 
(2)  the  sharing  provisions  of  a  value  engineering  incentive  clause 
included  in  any  type  of  contract. 

9.  Contract  Cost  Overrun  (Underrun)  -  A  net  change  in  contractual 
amount  over  (under)  that  contemplated  by  a  contract  target  price  (FPI 
contract),  estimated  cost  plus  fee  (any  type  cost  reimbursement 
contract)  or  redeterminable  price  (FPR  contract)  due  to  the  contractor's 
actual  costs  being  over  (under)  target  or  anticipated  contract  costs, 
but  not  attributable  to  any  other  cause  of  cost  growth  previously 
defined.  Offsetting  profit  or  fee  adjustments  attributable  to  cost 
incentive  provisions,  if  any,  shall  be  considered  in  determining  the 
net  contract  cost  overrun  (underrun) . 

A  cost  estimate  must  provide  visibility  to  higher- level  reviewers  and 
decision-makers.  A  better  presentation  and  backup  for  cost  estimates 
will  provide  greater  visibility  and  thus  help  to  identify  low  estimates. 
Reviewers  will  look  for  reasonableness  in  the  estimate.  Reasonableness 
is  based  on  a  judgment  made  by  comparing  cost  histories  of  current 
and  past  programs.  Since  it  is  accepted  that  a  price  cannot  be  placed 
on  an  unknown,  it  is  not  realistic  to  expect  the  first  cost  point  estimate 
made  on  a  system  which  is  in  the  conceptual  stage  to  be  one  hundred  per¬ 
cent  accurate.  The  uncertainties  arising  from  lack  of  data  and  new 
technology  will  never  be  completely  overcome. 
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CHAPTER  9 


COST  EFFECTIVENESS 


INTRODUCTION  1 J 

Cost  effectiveness  analysis  is  a  method  of  study  designed  to  reveal 
information  necessary  for  an  intelligent  appraisal  of  alternative 
courses  of  action.  In  most  cases  where  technology,  costs,  and  goals 
are  numerous  and  complex,  a  standardized  methodology  cuts  through  the 
irrelevant  features  of  a  study  and  exposes  the  relevant  aspects  for 
systematic  comparison.  The  objective  of  the  Army  management  process 
is  to  develop  weapon  systems  which  are  optimum  in  terms  of  performance 
capability  to  meet  currently  foreseen  needs  for  the  resources  applied. 

DISCUSSION 

Cost  effectiveness  is  based  on  the  economic  principle  that  all  material 
decisions  involve  the  choice  of  how  best  to  allocate  scarce  and  limited 
resources  among  unlimited  and  competing  requirements,  responsibilities, 
or  goals.  Even  in  the  most  basic  military  planning,  cost  effectiveness 
is  as  old  as  weapons  systems  themselves.  But  the  increasingly  complex 


1 /  The  subject  of  cost  effectiveness  analysis  is  one  with  wide  and 
complex  ramifications  in  the  determination  of  the  "best"  course  of  action 
from  several  alternatives.  Each  analysis  of  this  type  must  be  built  up 
from  a  consideration  of  the  particular  problem  at  hand.  Appropriate 
measures  of  effectiveness  will  differ  according  to  the  primary  mission 
of  the  force  or  equipment  being  studied.  The  costs  to  be  considered 
also  must  be  carefully  scrutinized  since  costs  determined  to  be  common  to 
all  systems  under  study  may  not  be  able  to  be  eliminated  (ignored). 

This  is  because  one  system  may  be  sensitive  to  the  common  cost  whereas 
the  other  systems  may  be  insensitive.  The  discussion  contained  in  this 
chapter  is  a  broad  overview  and  to  obtain  a  more  definitive  appreciation 
of  the  subject  the  reader  should  refer  to  the  several  references  (as 
well  as  others)  listed  below.  A  Primer  of  Cost  Effectiveness,  Research 
Analysis  Corporation,  AD  816  751;  Models  in  Cost-Effectiveness  Analysis, 
Research  Analysis  Corporation,  AD  622  109;  A  Beginning  to  Cost- 
Effectiveness  Analysis,  Research  Analysis  Corporation,  AD  422  374;  The 
Critical  Region  in  Relative  Cost-Effectiveness,  Resource  Management 
Corporation,  AD  820  657;  System  and  Cost  Effectiveness  Manual  for  System 
Developers,  Lockheed  Missiles  and  Space  Co.,  AD  867  397;  The  Total  Cost 
Approach  to  Cost  Effectiveness  Within  The  Logistics  Priority  System, 

AFIT,  AD  863  834;  and  System/Cost  Effectiveness  Notebook,  Communications 
and  Systems,  Inc.,  Vol  I  -  AD  854  651,  Vol  II  -  AD  854  652. 
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relations  between  weapons  systems  as  the  state-of-the-weapon-art  advances 
requires  a  simultaneous  advance  in  the  methodology  used  to  analyze  the 
concurrent  costs  and  effectiveness.  The  state  of  the  cost  effectiveness 
art  must  be  sufficiently  advanced  to  objectively  embrace  all  relevant 
aspects  of  possible  alternative  systems.  This  advance  in  analytical 
capability  is,  in  fact,  nothing  more  than  a  systematic  (organized)  set 
of  guidelines  to  follow  to  obtain  detailed  cost  effectiveness  studies 
of  alternative  projects.  These  studies  provide  benefits  through  the 
universal  dimensions  of  comparability  and  visibility,  exposing  the 
relevant  aspects  on  a  basis  which  warrants  genuine  comparison  without 
fear  of  oversight  or  mistake.  This  becomes,  then,  a  major  management 
tool  to  achieve  the  goal  of  controlling  the  selection,  development, 
and  utilization  of  a  system  in  an  optimum  manner  throughout  its  life- 
cycle.  The  organized  manner  by  which  the  manager  assembles  these  inputs 
is  cost  effectiveness  analysis  and  may  be  illustrated  in  outline  form 
as  seen  in  Figure  1. 

A  cost  effective  situation  can  be  defined  as  any  situation  where  the 
expended  resources  (funded  costs)  were,  in  fact,  effective  in 
achieving  the  project  goal.  But  for  every  defined  goal,  there  is 
usually  more  than  one  means  to  achieve  that  goal,  and  each  may  also 
be  cost  effective  as  defined  above.  The  problem  is  to  select  that 
method  which  is  the  most  cost  effective.  To  do  this,  the  goal  must 
first  be  specified  and  agreed  upon.  To  attempt  to  reach  a  goal 
without  first  clearly  specifying  the  goal  itself  would  be  inconceivable 
to  the  decision  maker,  so  every  plan  necessarily  includes  the  desired 
goal.  Then  the  expected  costs  associated  with  each  alternative  path 
must  be  developed  and  assigned  to  each  alternative.  This  now  brings 
the  alternative  costs  and  the  desired  goal  into  view.  This  describes 
a  simple  array  of  cost  effective  alternatives  without  considering 
the  possibility  of  tradeoff  analysis,  a  technique  which  raises  cost 
effective  techniques  to  a  higher  level  of  sophistication  which  will  be 
considered  later.  Figure  2  illustrates  a  simple  cost  effective 
relationship  between  three  linear  alternatives.  The  key  to  under¬ 
standing  cost  effectiveness  analysis  in  this  case  is  merely  deciding 
upon  the  desired  level  of  output.  Once  this  goal  has  been  selected, 
the  rational  economic  choice  is  clear.  If  output  level  M  is  selected, 
the  least-cost  alternative  to  achieve  that  output  is  Alternative  A. 

This  is  determined  by  Alternatives  C  and  B  each  having  a  higher  level 
of  total  costs  associated  with  that  level  of  effectiveness.  From 
this  we  have  arrived  at  a  relationship,  or  ratio,  between  cost  and 
effectiveness.  Notice  that  the  desired  effectiveness  has  been  held 
constant,  while  varying  the  cost  level  as  among  alternatives  A,  B  and  C. 
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Looking  closely,  it  is  seen  that  the  ratio  of  cost  to  effectiveness 
in  Figure  2  between  Alternative  A  and  Output  M  is  less  than  either 
Alternative  B  or  C.  Thus  Alternative  A  has  the  lowest  cost  effective 
ratio  for  Output  M  of  the  three  simple  alternatives  considered.  The 
ratio,  like  the  graphical  treatment,  also  enables  the  analyst  to 
identify  and  isolate  the  best  situation  from  among  those  which  are 
all  satisfactory,  that  is,  merely  good.  After  the  alternative  with 
the  lowest  cost  effective  ratio  has  been  identified,  the  decision 
maker  can  be  more  fully  informed  and  can  select  the  most  likely  means 
in  terms  of  efficiency  and  availability  to  achieve  his  goal.  It  is 
emphasized  again  that. in  constructing  these  ratios  that  effectiveness 
is  held  constant.  Otherwise,  inconclusive  results  could  be  reached. 

While  this  process  of  analysis  describes  the  most  logical  course  for 
optimizing  resource  use,  it  is  also  recognized  that  in  certain 
situations  the  starting  point  of  analysis  would  be  from  the  cost  side 
rather  than  the  output  side.  In  this  case,  the  decision  maker  would 
be  handed  a  list  of  the  resources  (expendable  costs)  at  his  disposal, 
and  his  task  would  then  be  to  maximize  his  output  (effectiveness), 
rather  than  minimize  his  costs.  In  this  case,  it  is  recognized  that 
the  best  a  decision  maker  can  do  is  to  "sub  optimize",  that  is,  to 
optimize  (maximize)  within  the  constraint  of  his  apportioned  resources. 

In  many  ways,  this  method  is  analogous  to  the  relation  between  the 
"grand  plan"  and  its  "sub  plans".  The  overall  scheme  is  decided  upon 
at  the  highest  levels,  and  a  rough  apportionment  of  budgeted  resources 
to  component  agencies  is  selected.  The  agencies  receiving  their 
budgets  must  coordinate  and  synchronize  their  movements  to  maximize 
the  effectiveness  of  their  share  of  the  "grand  decision". 

Finally,  in  situations  where  the  project  goal  is  not  clearly  visible, 
due  to  uncertainty  regarding  the  exact  nature  of  the  competition,  such 
as  enemy  threat,  another  variation  in  cost  effectiveness  techniques 
occurs.  In  this  case,  the  goal  itself  may  be  treated  as  not  fixed, 
and  subject  to  tradeoff  considerations.  There  might  be,  for  example, 
a  range  of  acceptable  goals,  going  from  least  acceptable  to  most 
acceptable  with  any  of  these  alternative  levels  of  output  acceptable, 
where  the  choice  would  be  less  clear  than  before.  Since  each 
alternative  goal  would  probably  have  alternative  and  different  cost 
estimates,  it  becomes  apparent  that  the  more  ambitious  goal  would  have 
a  higher  cost  than  the  next  most  ambitious  goal  and  so  on.  In  cases 
such  as  these,  it  is  probably  better  to  select  a  goal,  under  uncertainty, 
which  appears  to  be  most  likely  to  satisfy  the  conditions  of  competition, 
and  then  choose  the  least  cost  alternative. 
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SUMMARY 


At  best,  cost  effectiveness  analysis  provides  information,  not  policy. 

It  furnishes  needed  answers  to  relevant  questions  rather  than  justifying 
any  particular  goal.  In  essence,  it  seeks  to  outline  alternative 
paths  and  the  costs  which  would  be  incurred  along  each.  As  such,  it 
cannot  be  expected  to  make  a  decision,  but  rather  to  assist  the  decision 
maker  toward  an  enlightened  choice.  In  order  for  this  goal  to  be 
achieved,  all  the  data  used  to  estimate  costs  (cost  analysis)  and 
effectiveness  must  be  accurate  and  reliable. 
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CHAPTER  10 


A  NEW  TECHNIQUE  TO  ANALYZE  R&D  ESTIMATES:  STECPLOT  1/2/ 

THE  PROBLEM 

In  the  early  stages  of  designing  a  complex  weapon  system  there  is 
difficulty  in  estimating  the  eventual  cost  of  the  Research  and  Develop¬ 
ment  (R&D)  program.  This  problem  has  become  more  and  more  apparent 
as  the  Department  of  Defense  finds  its  early  estimates  on  most  weapon 
systems  at  the  root  of  cost  overruns.  The  following  reasons  largely 
contribute  to  this  problem: 

1.  Engineers  are  naturally  optimistic  about  solving  technological 
problems  of  a  new  system. 

2.  There  is  insufficient  identification  of  technical  and  manage¬ 
ment  risks  and  the  planning  required  to  reduce  these  risks. 

3.  There  are  changes  over  time  in  the  configuration  of  the  system 
being  developed,  usually  toward  a  more  complex  system,  due  to  initial 
inadequate  definition  of  the  requirements. 

4.  Appropriations  usually  don't  materialize  in  the  amount  and  with 
the  timing  desired  by  the  engineers. 

5.  The  program  stretches  out  because  of  technical  problems, 
funding  problems  and  too  optimistic  a  schedule  with  too  much  concurrency. 


1 J  Adapted  by  CPT  B.  J.  Kadets  and  CPT  C.  D.  Billings  from  Horace 
Schow's  "Macrobehavior  of  Development  Costs  and  Estimates  (STECPLOTS) , " 
Washington,  DC,  US  Army  Materiel  Command,  1968. 

2/  Research  and  development  cost  estimates  have  historically  been 
optimistic  resulting  in  cost  overruns  as  the  development  program 
progresses.  This  consistent  underestimating  is  caused  by  the  procedures 
of  making  an  estimate  and  the  development  framework.  A  new  technique 
to  evaluate  research  and  development  cost  estimates  is  presented  and 
described  with  a  case  study  of  Army  missile  development  programs.  The 
basis  of  the  evaluative  technique  is  the  analysis  of  the  trend  of 
research  and  development  cost  estimates  on  the  assumption  that  a 
median  level  of  past  disturbances  will  persist  in  the  future.  The 
technique  is  applicable  to  any  research  and  development  estimating 
procedure  where  data  on  past  performances  is  available  as  a  data 
base. 
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6.  The  cost  estimating  procedure  used  reflects  the  above  problems 
and  understandably  underestimates  costs. 

The  characteristics  of  the  R&D  process  make  cost  estimating  difficult  and 
cost  analysts  are  continuing  to  search  for  more  satisfactory  costing 
techniques.  The  most  frequently  used  technique  is  to  estimate  the  level 
of  effort  required  to  complete  the  objectives  of  the  R&D  program  within 
given  time  constraints.  This  approach  assumes  that  either  the  state-of- 
the-art  is  sufficiently  advanced,  or  can  readily  be  advanced,  to  the 
extent  necessary  that  some  estimable  amount  of  effort  will  achieve  the 
breakthroughs  required.  Apparently  this  technique  is  not  very  satisfactory 
since  R&D  estimates  tend  initially  to  be  low  as  shown  in  Table  I.  The 
data  in  Table  I  was  collected  from  Army  Management  review  reports  termed 
Army  Selected  Acquisition  Reports  (SAR's)  and  from  the  offices  of  Project 
Managers.  An  Army  R&D  program  is  financed  primarily  from  the  Research, 
Development,  Test  and  Evaluation  (RDT&E)  appropriation  account  and  several 
program  estimates  are  displayed  in  Table  I.  This  data  shows  that  the 
range  of  growth  in  the  RDT&E  estimates  was  from  12%  to  4257,  while  the 
average  rate  of  growth  was  1347.. 

In  R&D  Cost  Estimates  of  recent  weapon  systems,  Army  analysts  defined  the 
components  and  their  subcomponents  of  a  weapon  system  to  as  detailed  a 
breakout  as  was  possible  in  order  to  isolate  the  system's  components. 

Then  a  contractor  estimate  was  obtained  for  each  identified  component. 

This  systematic  breakdown  of  a  weapon  system  enabled  the  analysts  to 
identify  many  separate  items  for  the  first  time.  Where  the  contractor 
had  no  estimate  for  a  newly  discovered  component,  Army  engineers  made 
their  own  estimate.  In  certain  instances,  the  Army  analysts  believed 
that  the  contractor  estimates  at  the  component  level  were  optimistic  and 
raised  the  figure.  The  total  package  was  then  summed,  spread  by  planning 
factors  over  the  proposed  R&D  time  schedule,  and  presented  as  the  cost 
estimate  for  the  R&D  phase  of  the  system. 

Under  today's  environment,  Defense  cost  overruns  are  closely  scrutinized. 
Therefore,  cost  estimating  techniques  which  inherently  result  in  cost 
overruns  are  clearly  unacceptable.  The  methodological  problems  inherent 
in  this  process  need  to  be  examined  and  solved.  In  the  final  analysis, 
the  technique  described  above  used  contractor  estimates  at  the  lowest 
identifiable  component  level  which  were  modified  by  Army  analysts  where 
necessary.  It  is  assumed  that  the  contractor's  estimates  were  also  based 
on  a  levelf of-effort-funding. 


10-2 


TABLE  I 

RDT&E  ESTIMATE  GROWTH 


(Increment/Original  Est) 


System 

Original 

RDT&E 

Estimate  $M  1/ 

Current 

RDT&E 

Estimate  $M  1/ 

Percentage 
Growth  in 
RDT&E 

Estimate  % 

CHEYENNE 

125.9 

201.8 

60 

GAMA  GOAT 

5.1 

9.5 

86 

SHERIDAN 

32.0 

35.8 

12 

M6CA1E2 

6.1 

13.9 

128 

MBT-70  2/ 

101.4 

259.6 

156 

TACFIRE 

25.4 

69.7 

174 

LANCE 

132.1  3/ 

447.5 

239 

SAM-D 

554.7  4/ 

1,006.0 

81 

SHILLELAGH 

111.3 

152.6 

37 

DRAGON 

42.1  5/ 

95.4 

127 

TOW 

49.5  6/  " 

106.1 

114 

PERSHING  1A 

58.3  V 

116.9  9/ 

100 

IMPROVED  HAWK 

20.7  8/ 

108.6  10/ 

425 

Average  percentage  growth  in  RDT&E  estimate 

^Increment /Original  Estimate)  134 


1/  31  Mar  70  Selected  Acquisition  Report  unless  otherwise  indicated. 

2/  US  share  of  Joint  R&D  program. 

3/  April  62  Missile  B  Weapon  System  Proposal. 

4/  August  65  Trade  Off  Study. 

5/  March  65  MAW  Technical  Development  Plan. 

,6/  September  62  Technical  Development  Plan. 

V  September  65  Program  Change  Request. 

8/  December  62  Technical  Development  Plan. 

£/  January  69  Project  Manager  Master  Plan. 

10/  April  70  Life  Cycle  Cost  Study. 
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This  paper  presents  a  new  technique  to  evaluate  R&D  cost  estimates  that 
are  constructed  by  the  work  breakdown  method  described  in  the  preceding 
paragraphs.  This  analytical  tool,  called  STECPLOT  analysis,  evaluates 
RDT&E  cost  estimates  through  a  macroeconomic  view  of  the  estimate.  The 
teem,  STECPLOT,  arose  from  a  "Study  of  Trends  and  Escalation  of  £osts."!/ 

THE  TECHNIQUE 

The  basis  of  the  STECPLOT  technique  lies  in  the  analysis  of  the  trend  of 
RDT&E  Cost  Estimates  made  during  the  R&D  phase  of  various  weapon  systems. 
A  study  of  various  weapon  systems  indicates  general  regularities  in  the 
RDT&E  estimates  as  the  R&D  phase  moves  toward  completion.  These 
regularities,  which  form  the  basis  of  the  technique,  can  be  expressed 
graphically  by  using  a  graph  whose  axes  are  normalized  to  a  1007.  scale 
as  shown  in  Figure  1.  The  abscissa  represents  a  development  program's 
time  period  and  the  ordinate  represents  the  program's  expenditure;  both 
expressed  percentage-wise.  Through  this  normalization  procedure  R&D 
programs  of  different  lengths  and  costs  are  comparable  since  both  costs 
and  time  are  expressed  in  percentage  terms. 

An  important  assumption  in  using  the  STECPLOT  approach  is  that  the 
particular  system  being  estimated  follows  the  trend  of  previous 
systems.  This  allows  evaluation  of  the  status  of  the  system  being 
considered  on  a  comparative  basis  with  the  history  of  other  systems. 

It  should  be  noted  that  this  comparison  is  not  always  valid.  Systems 
may  not  follow  previous  trends  for  reasons  peculiar  to  a  particular 
system.  Therefore,  complete  dependency  on  the  technique  without  further 
analysis  is  not  desirable.  However,  considering  our  present  inability  to 
forecast  the  occurrence  of  the  causes  of  cost  growth  and  their  impact, 
the  assumption  that  a  median  level  of  past  disturbances  will  persist  in 
the  future  is  a  reasonable  assumption. 

Past  disturbances  of  R&D  programs  are  reflected  in  the  analysis  via  time 
and  cost  variables. 

1)  T  =  Total  development  time  from  beginning  tn  completion; 

2)  t  =  Elapsed  time  of  the  development  program,  o  t  T; 

\ 

3)  e  =  An  estimate  of  total  RDT&E  cost,  R,  made  at  a  point  in  time 

with  elapsed  time,  t,  o  <  t  ^  T; 

4)  c  =  Known  cumulative  RDT&E  monies  expended  upon  completion  of 

the  development  phase. 


1/  Horace  Schow,  Macrobehavior  of  Development  Costs  and  Estimates 
ISTECPLOTS),  Washington,  DC:  US  Army  Materiel  Command,  1968. 


{ 


0 


10-4 


5)  R  =  The  known  amount  of  RDT&E  costs  expended  upon  completion  of 
the  development  phase. 

Note  that  T  is  a  known  time  for  systems  that  have  completed  development, 
but  an  estimate  of  total  time  for  systems  that  have  not  completed  develop¬ 
ment.  The  variable  e  is  known  from  the  beginning  of  the  program,  t  m  o, 
to  the  end  of  the  program,  t  =  T,  for  systems  that,  have  completed  develop¬ 
ment  and  is  the  actual  expenditure  upon  completion  of  development,  e  =  R 
when  t  =  T.  For  systems  that  have  not  completed  development,  the  "e's" 
are  know,  from  the  beginning,  t  =  o,  to  t  =  present  time  where  t  is  still 
less  than  T. 

In  terms  of  the  above  variables,  the  abscissa  of  the  graph  is  the  variable 
t/T,  or  percentage  of  development  time  completed.  The  ordinate  is  a  cost 
percentage;  therefore,  it  can  express  either  the  accuracy  of  the  RDT&E 
estimate,  (e/R),  at  an  elapsed  time,  t/T,  or  the  percentage  of  total  RDT&E 
monies  expended,  (c/R),  at  an  elapsed  time,  t/T. 

The  first  curve  considered  is  e/R  as  a  function  of  t/T.  This  curve  can 
be  labeled  the  "accuracy  curve."  It  indicates  how  accurately  the  estimator 
estimated  the  eventual  total  RDT&E  costs  at  different  normalized  points 
in  time.  An  example  will  suffice  to  show  how  this  curve  is  derived. 

Suppose  system  A  completed  its  development  phase  after  3  years  and  a  cost 
of  $100  with  the  estimates,  e,  of  RDT&E  cost,  R,  made  over  the  3  years  as 
shown  in  Table  II: 


TABLE  II 

EXAMPLE  OF  RDT&E  ESTIMATES  OVER  TIME 


t  (years)  012  3 

e  at  each  t  $30  $50  $90  $100 


T  =  3  years  and  R  =  $100 

From  this  information  accuracy  factors  can  be  derived  at  various  points  of 
the  R&D  program  as  shown  in  Table  III: 

TABLE  III 

EXAMPLE  OF  ACCURACY  FACTORS  OVER  TIME 

t/T  0 

e/R  .30 


.33 

.50 


.67  1.00 
.90  1.00 


The  time  ratio,  t/T,  indicates  the  percentages  of  the  R&D  program  time 
completed  when  an  estimate  was  made  and  the  accuracy  factor,  e/R, 
indicates  what  percent  that  estimate  was  of  the  eventual  cost.  For 
example,  when  the  R&D  program  began,  the  RDT&E  estimate  was  $30,  but 
the  eventual  cost  was  $100;  therefore,  at  t/T  =  0  the  accuracy  factor 
was  $30/$100  =  .30.  When  the  program  was  2/3  completed,  at  the  end 
of  the  2nd  Year,  the  estimate  was  $90,  but  the  eventual  cost  was  $100; 
therefore,  at  t/T  =  .67  the  accuracy  factor  was  $90/$100  =  .90. 

Plotting  e/R  as  a  function  of  t/T,  Figure  2  is  obtained  from  Table  III. 


J 

t 


NORMALIZED  TIME  t/T 
GRAPH  OF  TABLE  III 
FIGURE  10-2 

At  337,  completion  time,  the  estimator  was  507.  accurate  in  his 
estimate  of  final  RDT&E  cost.  Interpolating,  at  507.  completion,  it 
is  predicted  that  the  estimator  would  have  been  757.  accurate. 

The  second  curve  considered  is  the  distribution  of  the  actual  cumulative 
expenditures  as  a  function  of  time.  That  is,  c/R  as  a  function  of 
t/T.  For  example,  assume  System  Ars  expenditures  as  shown  in  Table  IV. 

TABLE  IV 


EXAMPLE  OF  CUMULATIVE  RDT&E  EXPENDITURES  OVER  TIME 


t 

0 

1 

2 

3 

c 

$0 

$30 

$80 

$100 

U 
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From  this  information  expenditure  factors  can  be  derived  at  various 
points  of  the  R&D  Program  as  shown  in  Table  V. 


TABLE  V 


EXAMPLE 

OF 

EXPENDITURE 

FACTORS 

OVER  TIME 

t/T 

0 

.33 

.67 

1.00 

c/R 

0 

.30 

.80 

1.00 

Table  V  shows  that  at  the  beginning  of  the  R&D  Program  zero  dollars  have 
been  expended  but  that  by  the  time  the  program  is  2/3  complete  80%  of  the 
RDT&E  funds  have  been  expended.  Plotting  c/R  as  a  function  of  t/T, 

Figure  3  is  obtained  from  Table  V. 


NORMALIZED  c 
COSTS  R 


1.00 


t/T 

NORMALIZED  TIME 

GRAPH  OF  TABLE  V 
FIGURE  10-3 


If  we  combine  Figures  2  and  3  we  obtain  Figure  4. 


1.00 


NORMALIZED  e  ,c  .75 

COSTS,  R  5R 

.50 

.25 


0  .25  .50  .75  1.00 

t/T 

NORMALIZED  TIME 

EXAMPLE  OF  ACCURACY  AND  EXPENDITURES  FACTORS  OVER  TIME 

FIGURE  10-4 


Figure  4  indicates  how  well  System  A’s  actual  RDT&E  costs  were 
estimated  as  well  as  how  expenditures  were  distributed  over  time. 

The  next  section  details  how  the  ideas  presented  in  this  section  are 
used  in  estimating  RDT&E  costs  for  systems  still  in  development  stages. 

EXPLANATION 

There  are  many  reasons  why  a  development  program  costs  more  than 
originally  estimated.  There  may  be  simple  underestimating,  program 
stretchout,  changes  in  technical  characteristics,  technical  problems, 
and/or  relocation  of  the  project.  The  above  reasons  are  common  to  most 
systems  and  are  taken  implicitly  into  account  in  the  derivation  of  the 
curves.  As  mentioned  previously,  any  abnormalities  in  a  system  under 
study  should  be  taken  into  consideration  when  using  this  technique. 

An  accuracy  curve  was  derived  from  studies  done  by  the  Cost  Analysis 
Office,  Headquarters,  Army  Materiel  Command,  for  systems  that  have 
completed  their  development  phase.  The  curve  was  calculated  by 
"averaging"  the  individual  accuracy  curves  for  a  number  of  similar 
systems.  The  expenditure  curve  was  likewise  calculated.  (Note:  The 
variance  among  the  individual  curves  should  also  be  studied  for  greater 
insight.)  Table  VI  provides  the  accuracy  and  expenditure  factors  over 
time  for  eight  Army  missile  systems.  The  accuracy  and  expenditure 
curves  derived  for  systems  having  completed  development  are  now  used  to 
estimate  the  time  and  costs  for  systems  6till  in  development.  The 
main  assumption  is  that  the  developing  system  will  follow  the  same 
general  experience  pattern  of  estimates  and  costs  as  the  sample  of 
systems  that  have  completed  development. 

Assume  System  A  has  been  in  its  development  phase  for  t  years,  and  to 
date,  c  dollars  have  been  spent.  At  time  t  the  estimate  for  total 
RDT&E  cost  and  time  is  e0  dollars  and  T]_  time  respectively.  At 
normalized  time,  t/T^  =  Xj_,  the  accuracy  curve  indicates  that  at  this 
point  in  time  the  accuracy  factor  is  y^,  o  y^  -«C  1.00  (in  terms  of 
%,  y^  x  100).  Therefore  ej_,  an  estimate  of  eventual  RDT&E  cost,  would 
be  e^  =  eQ/y^.  However,  e^  is  based  on  the  estimate  of  time  T^  which 
is  probably  optimistic  and  will  experience  slippages.  Computing  c/e^, 
the  cumulative  expenditures  to  date  over  the  estimate  of  eventual 
RDT&E  cost  and  using  the  expenditure  distribution  curve,  find 
X2  =  t/T2»  0  <  ^2  <T  1.00,  that  corresponds  to  c/e^.  Then  T2  = 
equals  a  new  estimate  of  time  to  program  completion.  Using  T2  as  the 
time  estimate,  t/T2  gives  a  new  accuracy  factor,  y2»  from  the  accuracy 
curve.  Then  e2  =  ei/y2  equals  a  third  cost  estimate  that  corresponds 
to  T2.  Repeat  this  procedure  until  the  time  and  cost  estimates  begin 
to  converge.  The  judgment  of  the  analyst  familiar  with  the  system  is 
used  to  determine  when  to  terminate  the  process. 


average 


OHniolON  00  0vOl§ 


”SSSS88888g 


SSSSSSSSggo 


°  SS8«8S888g 


ssg§sssg»8o 


-  SSSSSgggggg 


SSSRSRSg-gg 


O.OO^NSfN^olg 


° SSKSSSSgggg 


H  ’■!  H  r( 


OOH^g^gHNO 


u* 

>  ° 

I  s 

a  s 


SSSSSgggsgg 


§S35SSggS!S;g 


335SK888888 


SS2SS5?:SS2g 


£<  SS3S2S88888 


3  So2S5§ggsgg 

**••••« 


8§°go o 


10-10 


In  the  next  section,  a  numerical  example  will  be  given  to  further 
explain  the  process. 

EXAMPLE 

Figure  5  shows  a  typical  accuracy  curve,  e/R,  and  cumulative 
expenditure  distribution  curve,  c/R.  These  curves  are  derived  from 
systems  that  have  completed  development  and  all  variables  are  known. 

Note  that  the  accuracy  curve  indicates  that  before  development  begins, 
i.e.,  at  time  zero,  the  cost  estimate  of  a  particular  system  has  been 
on  the  average  407.  underestimated}  or,  in  other  words,  the  final 
development  cost  has  resulted  in  an  increase  of  2  1/2  times  the  original 
estimate.  The  following  hypothetical  example  explains  numerically  how 
the  STECPLOT  is  used  as  an  aid  in  analyzing  RDT&E  costs. 

System  A's  development  phase  has  been  ongoing  for  5  years.  The 
estimate  of  final  RDT&E  cost  is  currently  e0  =  $133  million  and  the 
final  time  to  completion  estimate  is  T^  =  10  years.  Table  VII  shows 
the  systems  expenditures  to  date  by  year. 

TABLE  VII 

SYSTEM  A  EXPENDITURES  TO  DATE 


t  (years)  0 

1 

2 

3 

4 

5 

c  =  Cumulative  $0 

RDT&E  ex¬ 
penditures  in 
millions  (dollars) 

$5 

$15 

$35 

$60 

$85 

T^  =  10  years 

eQ  =  $133M 

\ 

The  cumulative  expenditures  at  time  t  =  5  is  c  =  $85  million.  The 
steps  in  the  procedure  are  given  below,  shown  graphically  in  Figure  6 
and  summarized  in  Table  VIII. 

TABLE  VIII 

_ TO _ T1 _ T2 _ T3 _ T4 _ 

Estimated  R&D 
Program  Length 

(Years)  10*  10  11.6  12.5  12.8 

Estimated  Final 

Costs  (million  $)  $133*  $146  $158  $166  $168  1 


*  ORIGINAL  ESTIMATE  AND  CURRENT  ESTIMATE 
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ACTUAL  DEVELOPMENT  ACCURACY  AND  EXPENDITURE  CURVES 

Figure  10-5 


STEP  1  t/T^  —  5/10  =  .50  =  Xi ;  therefore,  =  Accuracy  Factor  =  .91 

el  ~  —  —  —  $146  million  for  10  years 

yi  .91 


Based  on  the  accuracy  curve,  the  estimate  of  50%  program  completion  has 
historically  been  91%  accurate.  Thus  the  final  cost  is  estimated  to 
be  $146  million  for  the  estimated  10  year  program. 


STEP  2 

a)  c_  $  85 

ej  $146 


.58,  which  means  that  58%  of  development  funds 
have  been  expended. 


b)  Using  the  expenditure  distribution  curve,  the  percentage  of  time 
completed  for  expenditure  of  58%  of  development  funds  has  historically 
been  43%,  which  means  the  development  program  is  not  507.  completed. 


X£  =  Time  Factor  =  .43  =  t/T2»  where  T2  is  the  new  time 
estimate.  Therefore,  T2  =  t/X2  =  5/. 43  =  11.6  years,  which  implies  a 
slip  of  1.6  years  in  the  R&D  Program. 

c)  At  t/T2  =  .43,  Y2  =  Accuracy  Factor  =  .84;  therefore,  e2  - 
e0/y2  —  $133/. 84  =  $158  million  for  11.6  years;  where  e2  is  the  estimate 
of  final  RDT&E  costs  based  on  the  new  time  schedule. 


STEP  3 

a)  c/e2  =  $_85  =  ,54  0f  rdt&E  funds  have  been  expended. 

$158 

b)  Then  t/T^  =  X3  =  .40  (Means  the  R&D  Program  is  407.  complete). 

Tt  =  t/X^j  =  5/. 40  =  12.5  years,  implies  that  the  R&D  program 

will  not  slip  1.6  years  but  2.5  years. 

c)  t/T,  =  .40,  yg  =  .80.  Therefore,  e^  =  eQ/y3  =  $133/. 80  =  $166 
million  for  12.5  years. 

STEP  4 

a)  c/e3  =  $85 / $1 66  =  .51  of  R&D  funds  have  been  expended. 

b)  Then  t/T^  =  X^  =  .39  (Means  the  R&D  Program  is  397.  complete). 

T^  =  t/X^  =  5/. 39  —  12.8  years,  which  implies  that  the  R&D 

Program  will  not  slip  2.5  years  but  2.8  years. 
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c)  t/T^  =  .39,  y^  =  .79;  therefore,  e^  =  ej =  $168  million 
for  12.8  years. 

At  this  point,  the  process  is  arbitrarily  stopped  since  it  appears  the 
differences  are  becoming  quite  small. 

Table  VIII  indicates  the  range  of  costs  and  time  estimates  for  the 
R&D  phase  of  System  A.  Even  without  a  time  slip  the  program  costs 
have  been  underestimated  by  $13  million.  However,  a  slip  of  2.8  years 
is  possible  and  RDT&E  costs  of  $35  million  greater  than  presently 
estimated  are  possible. 

Establishing  a  point  estimate  for  System  A  involves  somewhat  the  judgment 
of  the  analyst  familiar  with  the  system.  However,  an  approach  that 
may  assist  him  in  this  estimate  can  be  followed. 

If  it  is  assumed,  at  first,  that  the  time  estimate  of  10  years  and 
the  accuracy  factor  of  .91  are  correct,  then  the  cost  estimate  is  $146 
million. 

TABLE  IX 


Time  t  Years 

0 

1 

2 

3 

4 

5 

Cumulative  Expenditures,  c 

$0 

$5 

$15 

$35 

$60 

$85 

t/10  years 

0 

.1 

.2 

.3 

.4 

.5 

c/$146  million 

0 

.03 

.10 

.24 

.41 

.58 

The  last  two  rows  of  Table  IX  are  the  calculations  of  the  cumulative 
expenditures  over  time  of  System  A  based  on  a  total  RDT&E  cost  of 
$146  million  and  a  R&D  Program  length  of  10  years.  This  curve  (System 
A»s  current  expenditure  distribution  curve)  is  plotted  and  compared  to 
the  average  expenditure  distribution  curve  (See  Fig  7).  In  this  case, 
the  curve  is  below  and  to  the  right  of  the  average  distribution  curve. 

It  will  be  observed  that  the  position  of  System  A's  distribution  curve 
is  determined  by  the  estimates  of  total  time  and  total  cost.  The  curve 
will  move  up  and  to  the  left,  that  is  closer  to  the  average  distribution 
curve,  if: 

1)  The  total  time  estimated  is  increased  relative  to  the  total 
cost  estimate 

or  if: 
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CURRENT  EXPENDITURE  DIS 


2)  The  total  cost  estimate  decreases  relative  to  the  total  time 
estimate. 

These  facts  can  be  seen  In  Figure  7.  If  It  Is  assumed  that  the  original 
cost  estimate  of  System  A,  e0,  Is  low,  as  would  be  Indlcateo  by  the 
accuracy  curve,  then  It  can  be  Inferred  that  a  higher  estimate  Is  a 
better  approximation.  Given  this  assumption,  the  cost  estimate,  e 3  = 

$166  million  and  time  estimate,  T  =  12.5  years,  which  shows  a  distribution 
curve  quite  close  to  the  average,  may  be  a  good  point  estimate  of  the 
system's  development  program.  (Possibly  T4,  e4  is  even  better.) 

QUALIFICATION S  AND  EXTENSIONS 

The  main  assumption  used  in  STECFLOT  analysis  is  that  the  system  under 
study  will  follow  on  the  average  the  regularities  of  systems  that  have 
completed  the  R&D  cycle.  Considering  our  inability  to  forecast  the 
occurrence  of  cost  growth  causes  and  their  cost  impacts,  the  assumption 
that  a  median  level  of  past  program  disturbances  will  persist  in  the 
future  is  a  reasonable  assumption.  This  is  a  macroeconomic  approach. 

In  the  actual  use  of  the  STECPLOT  no  attempt  is  made  to  isolate  specific 
reasons  why  these  regularities  occur;  they  are  taken  into  account  at 
the  "MACRO"  level. 

The  analyst  should  be  aware  of  the  deficiencies  in  this  assumption. 

The  systems  under  study  may  have  characteristics  that  have  not  been 
encountered  in  any  previous  systems,  and  these  characteristics  may  ha'/2 
a  great  influence  on  the  system  cost.  The  correct  use  of  the  STECPLOT 
in  this  case  would  be  to  isolate  those  parts  of  the  system  that  do  have 
a  historical  basis;  and  those  parts  that  don't  might  be  analyzed 
separately.  Another  point  of  consideration  is  that  the  actual  cost 
estimators  (as  distinguished  from  cost  analysts)  may  become  aware  of 
the  analyst's  use  of  the  STECPLOT.  Consequently,  his  estimates  may 
consciously  or  unconsciously  take  this  fact  into  account,  and  his 
resulting  estimates  may  be  higher  or  lower  than  they  would  be  in  his 
naivetfe.  The  STECPLOT  may  then  be  self-defeating  and  that  is  not 
necessarily  a  deficiency.  Since  past  history  has  indicated  RDT&E  costs 
to  be  usually  underestimated,  the  estimator  may  work  harder  to  upgrade 
his  work. 

At  any  rate,  a  known  deficiency  is  to  use  the  STECPLOT  solely  without 
other  further  analysis  by  someone  not  experienced  with  the  system. 

Much  study  is  required  to  make  the  STECPLOT  a  more  valuable  tool  in 
cost  analysis.  It  is  important,  for  example,  to  study  the  variance 
of  the  average  curves.  If  the  variance  is  relatively  large,  the  curve 
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use  is  negated  proportionately.  It  may  be  possible  using  variances  to 
determine  statistical  confidence  levels  of  analysis.  It  also  may  be 
possible  to  solve  the  plot  analytically  since  the  curves  approximate  known 
functional  shapes. 

CONCLUSION 

It  should  be  repeated  that  the  STECPLOT  technique  is  an  analyst's  tool? 
an  aid  in  studying  costs  at  the  macroeconomic  level.  The  main  assumption, 
stated  many  times  in  the  text,  must  always  be  borne  in  mind.  When  these 
points  are  considered  the  STECPLOT  has  proven  to  be  a  valuable  tool  in 
obtaining  more  realistic  cost  estimates.  In  fact,  a  number  of  Army 
systems  have  shown  a  marked  degree  of  consistency  in  following  the  trends 
discovered  from  previous  systems. 
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APPENDIX  A 


Modeling  and  Simulation: 

I.  A  model  is  a  representative  of  the  reality  of  an  object,  situation, 
condition,  or  operation  under  study.  Its  primary  function  is  "explanatory" 
rather  than  descriptive.  To  present  the  situation,  condition,  or 
operation  without  error  or  uncertainty  would  be  ideal.  However,  the 
problem  encountered  in  trying  to  construct  such  a  model  is  that  it 
isn't  known  at  the  start  what  is  relevant  and  what  is  superfluous. 

The  first  thing,  then,  in  attempting  to  build  a  model  is  to  discover 
the  relevant  factors.  Following  this,  the  model  builder  picks  those 
facts  which  are  or  can  be  quantified,  reduces  this  list  of  quantified 
factors  to  a  more  manageable  number  through  aggregation,  and  finally, 
spells  out  the  quantitative  relationships  between  the  relevant  elements. 
Successful  accomplishment  of  these  four  steps  will  give  the  analyst  a 
most  versatile  tool  for  use  in  his  mission  as  problem  solver.  Since 
models  are  representations  of  real  things  or  processes,  there  are 
cautions  to  be  observed.  First,  to  become  enamored  with  a  model  can 
lead  easily  to  making  all  sorts  of  inferences  about  the  problem  but  leave 
the  question  for  which  the  model  was  constructed  unanswered.  Second,  in 
order  to  impress  superiors  with  their  technical  competence,  analysts  may 
try  to  interject  structural  complexity  into  their  model  which  is 
unwarranted  by  the  type  of  data  available.  The  danger  here  is  that  an 
erroneous  conclusion  could  be  reached  -  either  the  wrong  answer  or  saying 
that  no  answer  can  be  reached.  Next,  trying  to  treat  all  aspects  of  a 
problem  simultaneously  can  lead  to  criticisms  that  relevant  facts  were 
left  out.  If  found  to  be  true,  and  the  analyst  attempts  to  satisfy  the 
objections,  the  model  becomes  bigger  and  more  complicated.  The  analyst 
must  maintain  focus  upon  the  real  question  being  asked  and  he  must  avoid 
trying  to  do  too  much  at  one  time.  Finally,  and  very  important,  he 
should  avoid  the  notion  that  there  is  such  a  thing  as  a  universal  model, 
that  is,  a  single  model  which  will  provide  answers  to  all  questions 
relative  to  a  given  activity.  Perhaps  the  basic  reason  for  the 
inapplicability  of  the  universal  model  concept  is  that  such  a  framework 
tends  to  cloud  or  eliminate  through  assumptions,  many  extremely  important 
problem  uncertainties.  These  uncertainties  must  be  explicitly  considered 
and  visually  treated  in  all  future  cost  analyses.  These  models  may  be 
classified  into  three  basic  types:  iconic,  analog,  and  symbolic. 

ICONIC  MODELS.  Iconic  models  look  like  the  object  which  they  purport 
to  represent.  They  result  from  the  scaling  down  in  size  an  object  such 
as  an  airplane  or  world  globe. 
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ANALOG  MODELS.  These  models  are  produced  by  a  convenient  transformation 
of  certain  properties  of  the  objects  represented.  An  electrical  circuit 
diagram  of  a  computer  is  an  analog  of  the  physical  system  of  equipment. 

SYMBOLIC  MODELS.  Symbolic  models  represent  the  characteristics  and 
relationships  of  the  original  problem  or  situation  through  the  use  of 
symbols.  This  form  of  model  does  not  bear  any  physical  resemblance  to 
the  original  thing  or  situation.  Mathematical. models  are  examples  of, 
symbolic  models. 

Mathematical  models  are  the  type  of  models  which  are  of  most  interest 
to  the  cost  analysis  community,  may  be  further  described  by  the  terms 
deterministic  (exact)  and  probabilistic  (stochastic).  The  former  is 
characterized  by  the  condition  that  chance  plays  no  role  in  arriving 
at  the  solution.  The  latter  type  is  one  where  chance  plays  a  major 
role.  These  types  of  models  involve  solutions  derived  through 
analytic,  numeric,  or  Monte  Carlo  processes.  Turning  specifically 
to  cost  models  as  the  subset  of  mathematical  models  with  which  we  are 
herein  concerned  several  cost  model  characteristics!./  should  be  noted: 

1.  The  total  resource  implication  of  the  systems  or  organizations 
to  be  costed  are  divided  into  three  major  cost  categories  that  portray 
their  time-phased  Mcradle-to-grave"  life  cycle. 

a.  Research  and  development  costs. 

b.  Initial  investment  costs. 

c.  Annual  operating  costs. 

2.  Each  of  these  major  cost  categories  is  composed  of  other 
subsets  of  categories  constructed  to  preclude  the  omission  of 
significant  cost  elements. 

3.  The  relation  of  each  element  and  category  is  spelled  out 
quantitatively. 

4.  The  cost  model  is  designed  in  such  a  way  as  to  assist  in  the 
analysis  of  the  cost  elements  that  have  the  greatest  impact  on  the 
total  cost  of  a  system  or  organization. 

5.  The  cost  model  in  its  final  form  usually  is  composed  of  a  set 
of  equations  that  give  in  quantitative  terms  the  type  of  cost 
information  desired. 


1/  Research  Analysis  Corporation  Technical  Paper  (TP-183),  An  Individual 
System/Organization  Cost  Model,  Jan  66. 
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From  a  consideration  of  these  characteristics,  a  cost  model  may  be 
defined  as  any  generalized  framework,  normally  involving  quantitative 
techniques,  used  to  estimate  the  relative  costs  of  competing  systems 
or  organizations.  The  design  of  these  models  depends  upon  the  data 
inputs  available  on  cost  and  quantities  of  physical  resources  and  the 
output  in  terms  of  cost  categories  desired  for  a  specific  study. 

Of  course  there  is  a  wide  diversification  of  cost  studies  and  estimates. 
Correspondingly,  there  is  a  wide  diversification  of  cost  models.  The 
simplest  form  of  a  cost  model  is  a  CER  or  cost  equation  such  as  may  be 
used  to  come  up  with  a  rough  estimate  of,  let's  say,  the  production 
cost  of  a  rifle. 

Cost  -  f(wt,  cal) 

A  most  comprehensive  and  detailed  type  of  model  would  be  required  to 
conduct  a  total  force  structure  cost  study  involving  several  mixes  of 
various  types  of  military  forces. 

Most  cost  studies  end  analyses,  however,  fall  in  between  these  two 
extremes  of  complexity.  As  a  result  of  this  large  diversification  of 
types  of  cost  estimates  and  studies,  there  is  a  wide  range  of  basic 
costing  approach/ philosophy ,  model  types,  and  types  of  computational 
,  equipment  which  cost  analysts  employ. 

The  basic  costing  approach  used,  such  as  the  level  of  cost  detail  to  be 
employed  and  the  establishment  of  accuracy  requirements,  is  dependent 
upon  the  nature  of  the  problem,  the  availability  of  time  and  competent 
personnel,  as  well  as  upon  the  availability  and  accuracy  of  relevant 
basic  information  and  data.  For  instance,  it  is  by  careful  consideration 
of  these  factors  that  one  would  decide  whether  to  use  3  detailed 
engineering  cost  approach  of  a  cost  estimating  philosophy  employing 
primarily  the  statistical  or  parametric  approach.  j 

In  making  this  decision,  it  is  well  to  bear  in  mind  Enthoven's  admonition 
to  systems  analysts  that  it  is  better  to  be  roughly  right  than  to  be 
precisely  wrong.  Saying  it  another  way,  the  degree  of  accuracy  sought 
and  the  level  of  detail  employed  should  be  consistent  with  the  type  and 
purpose  of  the  problem  as  well  as  the  degree  of  accuracy  and  the  level 
of  detail  of  available  data  and  other  cost  information.  Of  course,  the 
availability  of  time  is  also  a  most  important  factor  in  deciding  upon 
a  basic  costing  approach. 

This  matter  of  basic  costing  approach  is  a  very  very  important  one, 
since  it  is  so  fundamental  that  it  generally  determines  the  entire  mode 
and  tone  of  operation  of  a  cost  analysis  organization.  Stated  most 
simply,  this  philosophy  may  be  stated  as  follows: 
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Except  for  those  rare  instances  in  which  a  high  degree  of  accuracy 
is  mandatory  and  the  required  time,  information,  and  competent 
engineering  talent  are  available,  the  statistical  or  parametric 
approach  is  generally  considered  preferable  to  the  detailed 
engineering  costing  approach. 

PRINCIPLE  TYPES  OF  COST  MODELS 

Mathematical  cost  models  are  either  deterministic  or  stochastic.  A 
deterministic  model  treats  parameters  or  inputs  as  discrete  point 
estimates  while  a  stochastic  model  treats  input  parameters  as  random 
variables  so  that  resulting  costs  are  developed  in  the  form  of  a 
distribution  of  costs.  The  models  discussed  in  detail  later  on  are 
deterministic,  though  these  conceivably  could  be  converted  to  the 
stochastic  type  with  the  use  of  a  random  number  generator  and  inputs 
in  the  form  of  probability  distributions.  Use  of  the  stochastic 
model  is  one  way  to  handle  the  problem  of  cost  uncertainty. 

Three  types  of  computational  equipment  are  necessary  in  the  efficient 
operation  of  a  Cost  Analysis  Office.  These  are:  (1)  desk  calculators 
and  adding  machines;  (2)  desk  type  computers;  and  (3)  high  speed 
computers.  All  of  the  models  which  will  be  discussed  in  this  appendix 
were  designed  for  use  with  a  high  speed  computer.  In  most  cases,  they 
were  designed  to  develop  detailed,  yearly  life  cycle  costs  for  several 
mixes  of  several  different  weapons  per  mix. 

Although  there  is  a  wide  variety  of  cost  studies  and  analyses 
pertaining  to  Department  of  Defense  problems,  there  is  general  agree¬ 
ment  that  most  of  these  fall  into  one  of  three  basic  types.  These  are 

1.  Force  Structure  Cost  Analysis 

2.  Individual  Organization  Cost  Analysis 

3.  Weapon  System  Cost  Analysis 

Although  all  three  types  of  cost  analyses  attempt  to  determine  the 
resource  implications  of  alternative  courses  of  action,  they  do  so 
under  different  conditions.  The  third  type,  or  "Weapon  System  Cost 
Analysis",  is  generally  concerned  only  with  the  cost  of  specific 
weapon  systems  and  those  associated  costs  which  are  directly  related 
to  these  systems.  Army  tank  cost  problems,  for  instance,  might  be 
concerned  with  the  acquisition  costs  of  the  tank,  trainers,  and 
associated  equipment,  as  well  as  operational,  maintenance,  and 
training  costs.  A  weapon  system  cost  study,  for  a  given  type  of  tank, 
would  include  costs  for  crew  and  maintenance  personnel,  costs  for 


their  training;  as  well  as  costs  for  repair  parts,  POL  (i.e., 

Petroleum,  Oils  and  Lubricants),  and  overhaul  and  rebuild.  Unlike 
the  first  two  types  of  cost  analysis  problems,  Weapon  System  Costing 
does  not  include  the  costs  of  other  personnel,  equipments,  and 
facilities  which  may  be  part  of  the  same  army  organization  to  which 
the  tanks  are  assigned. 

In  regard  to  the  principal  difference  between  Force  Structure  and 
Individual  Organization  Cost  Analysis,  Dr.  Robert  Crosse  explains  this 
very  concisely  and  to  the  point  by  saying  ----  "Individual  Organization 
Cost  Analysis  determines  the  costs  of  a  given  alternative  as  if  it  were 
isolated  from  the  rest  of  the  force.  On  the  other  band,  Force  Structure 
Cost  Analysis  costs  the  alternative  in  the  context  of  the  total  force 
so  that  interrelationships  that  occur  among  the  organizational  elements 
of  the  force  may  be  realistically  appraised.  —  Consequently,  the  basic 
underlying  difference  between  the  two  types  of  organization  type 
costing  is  in  the  treatment  of  inventory  levels  and  the  availability 
of  assets  as  changes  in  the' force  occur.  Force  structure  cost  analysis 
attempts  to  quantify  these  asset  movements,  such  as  the  availability 
of  personnel  and  equipment  to  an  incoming  organization  as  another 
organization  phases  out  of  the  force." 

Such  organizational  type  cost  analysis  problems  may  pertain  to  the 
development  of  life  cycle  costs  for  a  completed  defense  organization, 
ranging  from  a  single  army  division  or  even  a  divisional  sub-unit, 
all  the  way  to  the  entire  U.  S.  Army,  Navy  or  Air  Force  ---  or,  it 
could  even  pertain  to  costs  for  the  entire  U.  S.  Defense  establishment. 

As  an  example,  consider  the  costing  of  a  specific  type  of  U.  S.  Army 
Division.  One  would  have  to  list  all  the  ordnance  and  quartermaster 
materiel  ordinarily  assigned  to  such  a  division.  Also,  one  would  have 
to  list  all  personnel  and  any  facilities  and/or  buildings,  if 
applicable.  The  acquisition  cost  of  all  materiel,  not  inherited  from 
some  other  organization,  would  have  to  be  computed.  Usually,  this  can 
be  done  by  applying  figures  from  standard  army  price  lists.  Yearly 
average  operational  and  maintenace  cost  factors  can  then  be  applied 
for  each  type  of  equipment  and  materiel  for  each  year  considered. 

Also  to  be  included  would  be  the  average  yearly  costs  of  all  consumables 
such  as  fuel.  Personnel  costs  would  include  yearly  pay  and  allowances, 
subsistence,  travel  costs,  and  any  applicable  training  costs. 

It  is  easy  to  see  that  a  model  developed  for  this  type  of  study  would 
indeed  be  very  large  and  would  require  a  large  number  of  inputs  and 
computations.  It  is  obvious  that  such  a  model  would  have  to  be  automated. 
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DESIGN  AND  USE  OF  WEAPON  SYSTEM  COST  MODELS 


As  a  result  of  the  large  diversity  of  types  of  cost  problems,  as  well 
as  the  variety  in  types  of  weapon  systems  and  other  Army  hardware 
which  an  Army  cost  analysis  office  may  have  to  deal  with,  one 
inevitably  is  faced  with  a  very  basic  problem  in  trying  to  decide 
upon  the  design  of  life  cycle  cost  models  to  apply  to  these  various 
problems  and  weapon  systems. 

It  appears  that  there  are  at  least  four  basic  approaches  to  this  cost 
model  design  problem.  These  are: 

1.  The  design  of  a  completely  flexible  model. 

2.  The  design  of  a  series  of  rigid  models  (each  designed  for  a 
specific  type  of  weapon  and/or  type  of  cost  study). 

3.  The  design  of  semirigid  models  (with  enough  flexibility  to 
be  applicable  to  several  types  of  weapons  and  studies). 

4.  A  concept  of  modular  constructed  models  (i.e.,  the  development 
of  a  library  of  submodels,  or  subroutines,  which  would  serve  as  the 
building  blocks  for  specific  life  cycle  cost  models). 

The  first  type  of  model  (i.e.,  the  completely  flexible  model)  is 
really  not  so  much  a  model  as  it  is  basic  framework  upon  which 
specific  models  may  be  built.  It  consists,  essentially,  of  a  computer 
program,  with  standardized  subroutines  and  standardized  input  and 
output  formats  which  facilitate  the  development  of  an  automated  life 
cycle  cost  model  to  meet  specific  cost  problems.  However,  use  of  this 
type  of  model  requires  the  development  and  programming  of  a  series  of 
cost  equations,  input  data  requirements,  and  requirements  for  the  output 
of  cost  summarizations.  The  basic  computer  program  merely  standardizes 
and  thereby  facilitates  the  conduct  of  these  tasks.  An  example  of  this 
type  of  model  is  the  series  of  Individyal  System/Organization  Cost 
Model  (ISOC)  developed  by  the  Research  Analysis  Corporation  for  the 
Department  of  the  Army.  This  type  model  has  certain  advantages  but 
requires  the  development  of  a  new  model  for  each  problem  presented. 

The  second  type  of  model  (i.e.,  the  design  of  a  tailor-make  model 
for  each  cost  problem)  is  self-explanatory  and  represents  a  very 
inefficient  approach. 

The  third  type  of  model,  the  semirigid  model,  with  sufficient 
flexibility  to  be  applicable  to  several  different  types  of  weapons 
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and  cost  problems,  is  a  feasible  approach  and  is  quite  efficient. 

However,  experience  shows  that  this  approach  has  certain  disadvantages 
which  may  be  overcome  only  by  making  the  model  more  complex  in  design 
and,  thereby,  making  it  more  cumbersome  and  difficult  for  analysts 
to  use  and  learn  to  use. 

In  trying  to  employ  each  of  these  philosophies  of  life  cycle  cost 
model  design  in  cost  studies  over  the  years,  the  analyst  arrives  at 
the  conclusion  that  none  of  the  first  three  types  of  models  is  really 
suitable  for  general  and  standard  application  to  work  in  most  cost 
analysis  offices.  It  appears  that  the  best  solution  is  the  fourth 
approach. 

This  fourth  basic  design  approach  is  termed  "Modular  Constructed  Models". 
According  to  this  concept,  a  library  of  cost  model  subroutines  (or  sub¬ 
models)  would  be  developed  as  the  need  for  each  one  became  necessary. 

These  would  be  subroutines  for  each  phase  of  the  life  cycle,  for  each 
basic  type  of  weapon  system,  for  various  output  formats,  various  methods 
of  treating  the  cost  uncertainty  problem,  and  for  many  other  purposes. 

The  idea  is  that  eventually,  with  a  varied  and  well-established  library 
of  such  subroutines,  an  analyst  will  be  able  to  readily  construct  an 
automated  life  cycle  cost  model  practically  tailor-made  to  apply  to  any 
cost  problem  be  a  judicious  selection  of  these  subroutines.  These 
subroutines  may  be  stored  on  disc  files  and  easily  retrieved  in  the 
required  order  by  a  few  punched  computer  cards. 

Actually  this  concept  has  been  demonstrated  in  the  latest  ICE  study  on 
the  Bushmaster  Machine  Gun.  ICE  is  the  acronym  for  a  series  of  com¬ 
prehensive  life  cycle  cost  studies  on  important  Army  Materiel  Command 
weapon  systems  and  stands  for  "Improved  Cost  Estimates". 

Why  is  the  Army  concerned  with  life  cycle  cost  models  and  why  is  it 
so  important  that  such  models  be  automated?  In  the  very  early  1960's, 
Robert  McNamara's  cost-effectiveness  and  planning,  programming,  and 
budgeting  approach  resulted  in  continual  demands  upon  the  US  Army 
Materiel  Command  and  its  Major  Subordinate  Commands  for  very  comprehensive 
cost  studies  involving  life  cycle  costs  for  several  alternative  weapon 
designs  or  courses  of  action.  These  comprehensive  cost  studies,  the 
demand  for  which  is  continually  increasing,  usually  involve  the  develop¬ 
ment  of  detailed,  yearly  life  cycle  costs  for  several  weapon  systems, 
one  weapon  system  with  several  different  quantities  (and/or  time  frames) 
or  a  combination  of  several  weapons  systems  and  many  different  quantities 
and  time  frames  per  weapon  system. 

In  the  early  1960's,  such  cost  studies  were  computed  with  desk 
calculators.  A  special  staff  of  many  experts,  calculator  operators, 


and  typists  would  be  assembled  in  a  large  room.  Large  cost  charts  would 
be  prepared.  Sometimes,  it  would  take  months  to  complete  a  study 
involving  several  alternative  courses  of  action,  and  the  resulting  costs 
for  each  alternative  would  be  based  upon  a  fixed  set  of  assumptions, 
weapon  quantities,  and  cost  factors.  In  the  process  of  conducting  such 
a  study,  so  much  time  had  to  be  devoted  to  calculations,  typing,  and 
preparation  of  the  final  report  that  those  in  charge  felt  compelled  to 
start  the  calculation  process  too  early  in  the  game,  before  the  problem 
had  been  sufficiently  formulated  and  before  a  proper  search  for  data 
could  be  conducted.  Of  course,  many  mathematical  errors  were  generally 
made,  either  by  those  conducting  the  computations  or  by  typists 
transcribing  the  results  of  calculations. 

As  if  these  problems  were  not  enough,  the  worst  problem  of  all 
usually  became  apparent  after  the  study  was  completed  and  the  special 
task  force  disbanded. 

Since  each  alternative  course  of  action  was  based  upon  one  set  of 
assumptions  and  one  set  of  numbers  as  input  data,  it  was  quite 
embarrassing  when  a  general  or  some  other  decision-maker  would 
disagree  with  the  choice  of  assumptions  or  numerical  input  data 
employed.  Sometimes,  a  slight  change  in  one  of  the  input  factors  or 
assumptions  could  upset  completely  the  relative  final  costs  of  the 
alternative  courses  of  action.  We  shall  see  later  that  a  single 
factor  can  control  the  final  results  and  conclusions  of  an  entire 
study. 

Needless  to  say,  this  type  of  an  approach  for  the  conduct  of  such 
comprehensive  cost  studies  was  not  only  inefficient  and  wasteful  of 
resources,  but  also  was  quite  inadequate  in  that  it  never  got  the  job 
done  properly. 

By  1964,  it  was  quite  obvious  to  DA  analysts  involved  in  cost/ 
effectiveness  studies  that  the  answer  to  the  problem  was  the 
development  of  automated,  mathematical  cost  models, 

II.  Simulation  is  defined  as  the  use  of  a  model  which  takes  account 
of  those  essentials  of  reality  which  are  significant  to  the  decision¬ 
making  objective.  A  decision  can  be  made  by  running  the  simulation 
a  number  of  times  for  different  alternatives,  if  necessary,  and 
comparing  results.  The  model  does  not  have  to  look  like  the  real 
environment  it  is  simulating,  but  it  must  give  the  same  results 
which  the  real  environment  would  yield.  Thus,  simulation  is  the 
technique  of  performing  sampling  experiments  on  the  model  of  the 


real  system  to  reach  a  decision.  The  experiments  are  performed  on 
the  model  rather  than  on  the  real  system  itself  only  because  the  latter 
would  be  too  expensive,  inconvenient,  and  time-consuming.  It  should 
be  obvious  then  that  for  a  simulation  experiment  to  be  successful,  it 
is  essential  that  the  simulation  model  abstract  the  major  characteristics 
of  the  real  world  problem. 

There  are  many  types  of  models  used  in  simulation  experiments.  Some 
of  the  more  common  examples  are  physical  duplications,  such  as  an 
aircraft  trainer;  contour  maps  to  study  geographical  areas;  a  layout 
diagram  of  a  factory;  charts  and  graphs;  mathematical  models;  etc. 

The  type  of  model  of  most  concern  for  this  discussion  is  the 
mathematical  model.  In  a  mathematical  model,  symbolism  in  terms  of 
variables  and  relationships  is  used  to  represent  and  describe  how  the 
factors  under  study  behave  in  reality.  Since  mathematical  models  are 
quite  often  very  complicated,  the  digital  computer  is  often  used  to 
carry  out  calculations.  As  a  matter  of  fact,  most  meaningful 
mathematical  models  must  use  the  computer  and  as  such  it  is 
Indispensable.  Some  of  the  examples  discussed  here  will  not  require 
the  computer  but  its  use  can  still  be  made  clear. 

Example  1:  A  game  is  played  in  which  a  fair  coin  is  flipped  until  the 
difference  between  the  number  of  heads  and  number  of  tails  is  3,  This 
ends  1  experiment  and  the  coin  is  tossed  again  for  second  and 
subsequent  experiments.  It  is  required  to  pay  $1.00  for  each  toss 
during  an  experiment  and  at  the  end  of  the  experiment,  the  player 
receives  $8.00,  Should  one  play  this  game? 

The  answer  could  be  calculated  analytically  to  determine  the  average 
(expected)  number  of  tosses  before  the  experiment  ends.  However,  it 
is  not  a  simple  solution.  One  could  toss  a  coin  for  a  half  an  hour 
and  observe  the  outcomes;  or  one  could  simulate  the  game  on  the 
computer  and  observe  the  results  by  that  method. 

At  this  point,  it  is  worth  noting  the  differences  between  the  analytic 
solution  and  the  simulated  solution.  The  analytic  solution  will  be 
exact  whereas  the  simulated  solution  will  always  depend  on  the  number 
of  times  the  experiment  is  performed  and  how  good  the  model  is.  Even 
then,  it  is  highly  unlikely  that  the  exact  solution  will  be  obtained. 

On  the  other  hand,  many  times  an  analytic  solution  is  almost  impossible 
or  very  costly  to  obtain  as  compared  to  the  closeness  and  convenience 
of  simulating  the  answer.  These  considerations  must  be  made  in  order 
to  determine  which  method  to  use. 
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To  illustrate  the  simulated  solution  to  this  example,  consider  the  fact 
that  most  computers  have  what  are  known  as  pseudo-random  number 
generators.  (Random  numbers  are  extremely  important  in  simulation 
and  will  be  discussed  in  much  more  detail  further  on.)  The  most  common 
algorithm  is  one  which  generates  the  numbers  from  0-9  and  each  has  a 
probability  of  1/10  of  occurring.  Therefore,  since  a  head  and  a  tail 
have  probabilities  of  1/2,  arbitrarily  designate  the  occurrence  of  the 
numbers  0,  1,  2,  3,  and  4  as  meaning  a  head  occurs;  and  5,  6,  7,  8,  9 
as  a  tail  occurring.  Thus,  generating  a  sequence  of  numbers  until  the 
number  of  numbers  5  or  above  is  three  more  than  the  numbers  4  or  below 
or  vice  versa  constitutes  the  end  to  one  experiment. 

Example  1  -  continued: 

8,  1,  3,  7,  2,  7,  1,  6,  5,  5,  7  7T,  4H 

T,  H,  H,  T,  H,  T,  H,  T,  T,  T,  T  11  flips  of  the  coin  were 

required 

Suppose  we  made  14  experiments  and  the  number  of  flips  until  each 
experiment  ended  was  11,  5,  5,  9,  7,  7,  5,  3,  17,  5,  5,  3,  9,  and  7. 

The  average  number  of  tosses  from  this  sample  calculates  to  be  7. 

Based  on  this  simulation,  one's  decision  would  be  to  participate  in 
the  game  because  he  would  in  the  long  run  expect  to  win  $1.00  per 
game.  It  is  interesting  to  note  that  the  analytic  solution,  which  is 
of  course  exact,  calculates  to  an  expected  number  of  tosses  of  9. 
Therefore,  a  player  would  expect  on  the  average  to  lose  $1,00  per 
game.  This  indicates  that  a  sample  size  of  14  for  this  simulation 
may  have  been  too  small.  Beyond  the  scope  of  this  instruction  is 
the  design  of  simulation  experiments.  I 

| 

Simulating  Risk  (Monte  Carlo  Techniques) 

Almost  all  economic  decisions  involve  information  that  is  not  certain; 
that  is,  risk  and  uncertainty  are  present.  Monte  Carlo  techniques, 
which  is  the  exotic  name  for  methods  using  random  numbers,  are  the 
methods  used  to  simulate  risk  in  situations  where  the  probability 
distributions  are  known. 

Random  number  generators  in  computers  have  been  previously  mentioned, 
and  the  computer  is  usually  the  medium  for  economic  problems.  As 
also  mentioned,  pseudo-random  number  generators  in  the  computer 
usually  consist  of  generating  the  numbers  from  0  to  9  (uniform 
distribution)  all  with  equal  probability  of  1/10.  Therefore,  Monte 
Carlo  techniques  include  methods  of  translating  the  uniform 
probability  distribution  into  other  probability  distributions  such 
as  the  normal  or  exponential. 
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Example  2: 

A  maintenance  shop  has  to  be  set  up  in  order  to  service  items  at  a 
particular  base.  Previous  data  indicate  that  the  number  of  items  that 
are  required  for  repair  per  day  and  the  time  to  repair  them  follow  the 
following  probability  distributions  respectively. 


#  items /day 

9  10  11  12 

Probability 

.4  .2  .3  .1 

b.  Time  for  repair  -  Exponential  Distribution:  P(X>K)  =  1  -  e 
where  X  =  time  to  repair  (hours),  K  =  given  #  of  hours  which  means  the 
probability  that  the  repair  job  X  takes  less  than  K.  hours  is  the 
expression  shown  (1  -  e  "^). 

The  question  is  "How  many  men  must  be  assigned  to  the  shop  so  that  there 
is  a  907.  chance  that  all  the  work  received  will  be  completed  in  an  eight 
hour  day?" 

Setting  up  this  simulation  experiment  is  not  very  difficult;  but  Monte 
Carlo  techniques  are  used.  To  generate  random  numbers  to  simulate  the 
number  of  items  for  repair  each  day,  use  the  same  method  as  shown  in 
Example  1.  Therefore 


Generating  random  numbers 

0,1, 2, 3 

4,5  6,7,8  9 

Means  X  items  for  repair  per  day 

19 

20  21  22 

To  simulate  the  time  needed  for  repair  of  each  item  is  more  complicated 
and  requires  some  theory.  Notice  that  X,  the  time  to  repair,  can  be 
(theoretically)  as  small  as  zero  and  as  large  as  infinity.  Therefore 
1  -  e  ~K  equals  0  when  K  is  zero  and  equals  one  when  K  is  infinity. 

This  makes  sense  because  the  probability  of  something  occurring  can 
never  be  less  than  zero  or  greater  than  one. 


-K 

The  graph  of  1  -  e  is  shown 

\,o\ - 

o  k 

and  note  that  1  -  e  is  always  between  zero  and  one.  The  computer 
random  number  generator  can  give  numbers  from  zero  to  one  also  by 
expeditious  placing  of  the  decimal  point.  Thus,  one  can  generate  two 
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random  numbers  1  and  6  and  this  can  represent  the  number  .16.  Since 
1  -  e  and  the  random  number  generators  are  between  zero  and  one,  set 
the  number  generated  on  the  computer  equal  to  1  -  e  "K.  For  example, 
if  .50  was  generated, 

set  .50  =  1  -  e  “K,  or  e”^  =  .50 

Therefore,  k  =  .70  since  e“‘™  =  ,50  which  means  the  first  item  requires 
7/10  of  an  hour  (42  minutes)  service  time.  Thus,  by  generating  a  number 
from  a  uniform  distribution  and  applying  it  to  the  equation  of  the 
exponential  distribution,  the  number  derived  is  simulated  to  come  from 
the  exponential  distribution.  The  above  is  an  illustration  of  Monte 
Carlo  techniques  to  simulate  the  occurrence  of  phenomena  that  follow 
given  probabilities. 

Example  2  (continued): 

It  should  now  be  apparent  how  to  arrive  at  the  answer  using  simulation 
techniques.  Suppose  it  was  decided  that  10  days  would  be  a  large 
enough  sample  for  this  problem  and  the  following  information  was 
generated. 


Day  No. 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Random  Number  Gener. 

5 

0 

0 

4 

9 

3 

2 

8 

3 

6 

No.  of  items  repaired 

20 

19 

19 

20 

22 

19 

19 

21 

19 

21 

The  generation  of  time  to  repair  for  each  item  on  each  day  by  using 
the  method  previously  described;  and  then  adding  the  times  for  each 
day's  number  of  items  yields 


Day  No. 

Total  Repair  Time  (Hrs) 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

36 

24 

31 

43 

37 

20 

29 

29 

30 

34 

Assuming  that  each  man  works  a  10  hour  day,  the  following  table  is 
derived: 


20-29 

30-39 

40-49 

Number  of  Days 

4  days 

5  days 

1  day 

7.  of  Total 

407o 

507.  . 

107. 

No.  of  men  required/day 

3 

4 

5 

Based  on  this  sample,  it  is  seen  that  5  men  were  needed  only  once  in 
the  ten  days  simulated.  Therefore,  it  could  be  concluded  that  4  men 
would  satisfy  the  requirement  of  work  completion  907.  of  the  time. 


u 
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Appendix  C 
GLOSSARY 


ACCRUED  EXPENDITURES:  The  monetary  amounts  of  goods  received,  services 
rendered,  expenses  incurred,  assets  acquired  (except  as  noted 
below),  construction  performed,  and  grants  earned  during  a  given 
period,  regardless  of  when  payment  is  made,  or  of  whether  invoices 
have  been  received.  The  portion  of  any  such  expenditures  which 
is  unpaid  at  a  given  point  in  time  is  a  liability.  The  portion 
of  payments  made  for  which  the  expenditures  have  not  accrued 
(such  as  advances  and  pre-payments)  is  an  asset.  As  noted  by 
the  Budget  Commission,  accrued  expenditures  should  not  be  confused 
with  program  costs.  Accrued  expenditures  measure  resources 
acquired,  while  program  costs  measure  resources  used. 

ALGORITHM:  An  orderly,  step-by-step  procedure  for  performing  a 

mathematical  operation  in  a  finite  number  of  steps.  The  1040  form 
is  an  algorithm  for  computing  personal  income  tax. 

ALLOTMENT:  An  authorization  granted  by  an  operating  agency  to  another 
office  to  incur  obligations  within  a  specified  amount  pursuant 
to  an  appropriation  or  other  statutory  provision  and  subject  to 
specific  procedural,  bookkeeping,  and  reporting  requirements. 

ALTERNATIVES:  The  means  by  which  objectives  can  be  attained.  They 
need  not  be  obvious  substitutes  for  one  another  or  perform  the 
same  specific  function.  Thus,  to  protect  civilians  against  air 
attack,  shelters,  "shooting”  defenses,  and  retaliatory  striking 
power  are  all  alternatives. 

ANALOG  METHOD  OF  COST  ESTIMATING:  A  graphical  method^of  cost 

estimating  which  is  based  on  historical  costs  that  are  too  limited 
to  allow  statistical  estimating  and  which  is  more  economical  to 
prepare  than  an  engineering  estimate.  Graphical  analysis  is 
often  helpful  to  more  clearly  understand  the  degree  of  relationship 
of  the  data  points.  The  analog  estimate  is  normally  adjusted  by 
deducting  historical  costs  of  components.  See  also:  Statistical 
method  of  cost  estimating;  Cost  estimating  relationship; 

Engineering  method  of  cost  estimating;  Regression  analysis. 

ANALYSIS  OF  VARIANCE  (ANOVA):  The  basic  idea  of  ANOVA  is  to  express 
a  measure  of  the  total  variability  of  a  set  of  data  as  a  sum  of 
terms,  each  of  which  can  be  attributed  to  a  specific  source  or 
cause  of  variation. 


APPORTIONMENT:  A  distribution  made  by  the  Bureau  of  the  Budget  of 

amounts  available  for  obligation  or  expenditure  in  an  appropriation 
or  fund  account  into  amounts  available  for  specified  time  periods, 
activities,  functions,  projects,  objects,  or  combinations  thereof. 
The  amounts  so  apportioned  limit  the  obligations  to  be  incurred 
or,  when  so  specified,  expenditures  to  be  accrued. 

APPRAISAL:  Impartial  analysis  of  information  conducted  at  each 

responsible  management  and  control  level  to  measure  the  effective¬ 
ness  and  efficiency  of  the  total  process  and  determine  preventive/ 
corrective  action. 

APPROPRIATION:  Ordinary  current  appropriations  (either  no-year  or  one 
or  more  years)  are  budgec  authorizations  granted  currently  by 
Congress,  both  to  incur  obligations  and  to  make  expenditures  in 
a  definite  specified  amount.  Has  excluded  contract  authorizations 
and  authorizations  to  spend  debt  receipts.  Under  the  Budget 
Commission's  recommendations,  these  latter  types  of  authorizations 
are  also  called  appropriations,  but  appropriations  to  liquidate 
contract  authorizations  would  not  be  counted  as  new  appropriations. 

AUTHORIZATION:  An  Act  of  Congress  which  authorizes  Federal  programs, 
obligations,  or  expenditures.  The  term  sometimes  refers  to 
basic  substantive  legislation  setting  up  a  program  or  an  agency, 
and  authorizing  appropriations  to  be  made  for  them,  but  not 
actually  providing  authority  to  spend.  See  also:  Appropriation. 

AVAILABILITY:  The  probability  that  the  system  is  operating 

satisfactorily  at  any  point  in  time  when  used  under  stated 
conditions,  where  the  total  time  considered  includes  operating 
time,  active  repair  time,  administrative  time,  and  logistic 
time. 

AVERAGE  OUTGOING  QUALITY  LIMIT:  The  average  maximum  fraction  defective 
leaving  an  acceptance  sampling  plan. 

BALANCE  OF  INTERNATIONAL  PAYMENTS:  A  systematic  record  of  the 

economic  transactions  of  a  country  during  a  given  period  which 
involve  a  transfer  of  currency  between  the  country's  residents 
and  the  residents  of  the  rest  of  the  world, 

BIAS:  An  unbalanced  range  of  error  such  that  the  average  error  is 
not  zero. 

BREAK-EVEN  POINT:  In  engineering-economic  studies,  the  point  at  which 
two  alternatives  become  equally  economical  by  altering  the  value 
of  one  of  the  variables  in  a  situation. 
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BUDGE^;  A  proposed  plan  by  an  organization  for  a  given  period  of 
time  reflecting  anticipated  resources  and  their  estimated 
expenditure  in  the  pursuit  of  objectives. 

BUDGETING:  Budgeting  is  the  process  of  translating,  planning,  and 
programming  decisions  into  specific  projected  financial  plans 
for  relatively  short  periods  of  time.  Budgets  are  short-range 
segments  of  action  programs  adopted  which  set  out  planned 
accomplishments  and  estimate  the  resources  to  be  applied  for  the 
budget  periods  in  order  to  attain  those  accomplishments. 

CERTAINTY:  The  state  of  absolute  confidence  in  which  outcomes  are 
sure  and  predestined. 

COEFFICIENT  OF  CORRELATION  (r):  A  measure  of  how  well  a  linear 

regression  line  (q.v.)  fits  the  data.  This  measure,  when  squared, 
equals  the  coefficient  of  determination  (r^)  which  is  a  measure 
of  the  proportion  of  the  total  variation  (squared  deviations  from 
the  average)  in  the  dependent  variable  explained  by  variation  in 
an  independent  variable.  A  high  coefficient  of  correlation  of 
0.9  indicates  that  the  least-squares  regression  of  the  dependent 
variable  on  the  independent  variable  accounts  for  81  percent  (the 
coefficient  of  determination)  of  the  variance  in  the  dependent 
variable.  The  limit  of  the  coefficient  of  determination  and  of 
the  coefficient  of  determination  and  of  the  coefficient  of 
correlation, is  1.0. 

CONFIDENCE:  The  degree  of  trust  or  assurance  placed  in  a  given  result. 

CONFIDENCE  INTERVALS:  A  measure  of  effectiveness  in  testing,  expressed 
in  quantitative  terms;  e.g.,  the  value  of  a  specific  factor 
(variable)  lies  within  a  specified  interval  957.  of  the  time. 

CONFIDENCE  LEVEL:  The  probability  that  the  true  value  of  a  parameter 
lies  within  a  stated  interval. 

CONSOLIDATED  CASH  BUDGET:  A  Federal  budget  which  shows  receipts  from 
and  payments  to  the  general  public,  i.e.,  all  non-Federal  Govern¬ 
ment  units,  including  trust  funds  and  some  Federal  government 
sponsored  enterprises.  A  deficit  shows  that  the  public  is 
accumulating  cash  or  government  securities.  This  budget  stresses 
the  financial  impact  of  the  overall  Government  program.  See 
also:  Budget,  Administrative  budget;  National  Income  Accounts 
budget. 
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CONSTANT  DOLLARS:  A  statistical  series  is  said  to  be  expressed  in 
"constant  dollars"  when  the  effect  of  changes  in  the  purchasing 
power  of  the  dollar  has  been  removed.  Usually  the  data  are 
expressed  in  terms  of  some  selected  year  or  set  of  years. 

CONSUMER  PRICE  INDEX:  A  measure  of  the  period-to-period  fluctuations 
in  the  prices  of  a  quantitatively  constant  market  basket  of 
goods  and  services  selected  as  representative  of  a  specific  level 
of  living.  Hence,  it  can  be  thought  of  as  the  cost  of  maintaining 
a  fixed  scale  of  living. 

CONSTRAINT:  A  resource  limitation,  which  may  be  specific  (e.g.,  the 

supply  of  skilled  manpower  or  a  particular  metal),  or  general  (e.g., 
total  available  funds). 

CONTINGENCY  ANALYSIS:  Repetition  of  an  analysis  with  different 

qualitative  assumptions  such  as  theater,  or  type  of  conflict,  to 
determine  their  effects  on  the  results  of  the  initial  analysis. 

CORRELATION:  In  a  general  sense  in  statistics,  correlation  denotes  the 
co-relation  or  covariation  between  two  variables. 

COST  ANALYSIS:  The  systematic  examination  of  cost  (total  resource 
implications)  of  interrelated  activities  and  equipment  to 
determine  the  relative  costs  of  alternative  systems,  organizations, 
and  force  structures.  Cost  analysis  is  not  designed  to  provide  the 
precise  measurements  required  for  budgetary  purposes. 

COST  CATEGORIES:  Three  major  program  cost  categories  are:  (1)  Research 
and  Development.  Those  program  costs  primarily  associated  with 
research  and  development  efforts,  including  the  development  of  a 
new  or  improved  capability  to  the  point  of  operation.  These  costs 
include  equipment  costs  funded  under  the  RDT&E  appropriations  and 
related  Military  Construction  appropriation  costs.  They  exclude 
costs  that  appear  in  the  Military  Personnel,  Operation  and  Main¬ 
tenance,  and  Procurement  appropriations.  (2)  Investment.  Those 
program  costs  required  beyond  the  development  phase  to  introduce 
into  operational  use  a  new  capability,  to  procure  initial, 
additional,  or  replacement  equipment  for  operational  forces  or  to 
provide  for  major  modifications  of  an  existing  capability.  They 
include  Procurement  appropriation  costs  and  all  Military 
Construction  appropriation  costs  except  those  associated  with  R&D. 
They  exclude  RDT&E,  Military  Personnel,  and  O&M  appropriation  costs. 
(3)  Operating.  Those  program  costs  necessary  to  operate  and 
maintain  the  capability.  These  costs  include  Military  Personnel 
and  O&M  appropriation  costs,  including  funds  for  obtaining 
replenishment  spares  from  stock  funds.  They  exclude  RDT&E  and 
Military  Construction  appropriation  costs. 
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COST  CURVE:  A  graphical  representation  of  the  relationship  of  cost 
to  another  variable,  such  as  output.  It  is  conventional  to 
construct  these  curves  with  costs  along  the  vertical  axis  and 
the  related  variable  along  the  horizontal  axis. 

COST  EFFECTIVENESS  ANALYSIS:  The  quantitative  examination  of 

alternative  prospective  systems  for  the  purpose  of  identifying 
a  preferred  system  and  its  associated  equipment,  organizations, 
etc.  The  examination  aims  at  finding  answers  to  a  question  and 
not  at  justifying  a  conclusion.  The  analytical  process  includes 
trade-offs  among  alternatives,  design  of  additional  alternatives, 
and  the  measurement  of  the  effectiveness  and  cost  of  the 
alternatives. 

COST  ELEMENTS:  The  subdivision  of  cost  categories  related  to  work 
areas  or  processes  performed  in  developing,  producing,  and 
operating  a  weapon/ support  system. 

COST  ESTIMATE:  The  estimated  cost  of  a  component  or  aggregation  of 
components.  The  analysis  and  determination  of  cost  of 
interrelated  activities  and  equipment  is  cost  analysis. 

COST  ESTIMATING  RELATION  (CER):  A  numerical  expression  of  the  link 
between  a  physical  characteristic,  resource,  or  activity  and  a 
particular  cost  associated  with  it;  e.g.,  cost  of  aircraft 
maintenance  per  flying  hour. 

COST  INFORMATION  REPORTING  (CIR):  A  uniform  system  for  collecting 
and  processing  cost  and  related  data  on  major  items  of  military 
equipment.  Its  purpose  is  to  assist  both  industry  and  government 
in  planning  and  managing  weapon  systems  development  and  production 
activities. 

COST  MODEL:  An  ordered  arrangement  of  data  and  equations  that  permits 
translation  of  physical  resources  into  costs. 

COST  SENSITIVITY:  The  degree  to  which  costs  (e.g.,  total  systems  costs) 
change  in  response  to  varying  assumptions  regarding  future  weapon 
system  characteristics,  operational  concepts,  logistic  concepts, 
and  force  mix. 

DECISION  VARIABLE:  A  variable  over  which  one  can  exert  some  control, 
whose  value  one  can  choose  as  a  result  of  a  decision.  The 
decision  variable  might  be  the  amount  of  fc>'J  one  must  eat  to 
satisfy  hunger.  If  the  relationship  between  the  values  of  the 
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decision  variable  and  the  level  of  goal  attainment  can  be 
defined,  one  can  then  find  the  value  of  the  decision  variable 
that  maximizes  the  attainment  of  the  goal. 

DEGREES  OF  FREEDOM:  Refers  to  the  size  of  the  sample(n)  less  the 

number  of  parameter  estimates  "used  up”  in  the  process  of  arriving 
at  a  given  unbiased  estimate.  .For  example,  to  estimate  the 
arithmetic  mean  needed  to  calculate  the  variance  of  a  population 
it  is  necessary  to  use  the  arithmetic  mean  of  the  sample,  thu9 
using  up  one  degree  of  freedom.  The  estimate  of  the  population 
variance  would  thus  have  n-1  degrees  of  freedom.  In  the  case  of 
regression  analysis,  one  degree  of  freedom  is  lost  for  each  of  ra 
variables;  thus  the  regression  variance  has  n-m-2  degrees  of  freedom. 

DIMINISHING  MARGINAL  UTILITY:  The  principle  that,  4s  the  rate  of 
consumption  of  a  good  is  increased,  a  point  is  reached  where 
additional  unit9  provide  less  and  less  utility.  4 

DIMINISHING  RETURNS:  An  increase  in  some  inputs  relative  to,  other 
fixed  inputs  will  cause  total  output  to  increase;  but  after  a 
point  the  extra  output  resulting  from  the  same  additions  of 
extra  inputs  is  likely  to  become  less  and  less.  This  falling 
off  of  extra  returns  is  a  consequence  of  the  fact  that  the  new 
11  doses”  of  the  varying  resources  have  less  and  less  of  the  fixed 
resources  to  work  with. 

DISBURSEMENTS:  The  amount  of  expenditure  checks  issued  and  cash 
payments  made,  net*  of  refunds  received. 

ECONOMETRICS:  The  branch  of  economics  that  uses  mathematics  and 
statistics  to  build  and  analyze  economic  models,  to  explain 
economic  phenomena,  and  to  estimate  valued  for  economic 
variables.  The  statistical  methods  used  are  especially  designed 
to  deal  with  time-series  data. 

ECONOMIC  ANALYSIS:  A  systematic  approach  jto  a  given  problem,  designed 
to  assist  the  manager  in  solving  a  problem  of  choice.  The  full 
problem  is  investigated;  objectives  and  alternatives  are  searched 
out  and  compared  in  the  light  of  their  benefits  and  costs  through 
the  use  of  an  appropriate  analytical  framework. 

ECONOMIC  LOT  SIZE:  The  cost-minimizing  size  or  order  to  buy  or  batch 
to  make. 
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ECONOMIES  OF  SCALE:  Efficiencies,  usually  expressed  as  reduction  in 
cost  per  unit  of  output,  that  result  from  increasing  the  size 
of  the  productive  unit, 

ECONOMY:  Using  the  least  amount  of  resources  to  attain  a  given  output 
or  fixed  objective, 

EFFECTIVENESS:  The  degree  or  amount  of  capability  to  accomplish  some 
objective(s).  Various  criteria  (e.g.,  targets  destroyed,  tonnage 
moved,  etc.)  might  be  used  to  provide  a  measure  of  this  amount  of 
capability, 

ENGINEERING  METHOD  OF  COST  ESTIMATING:  A  traditional  means  of  cost 

estimating  which  depends  on  a  well-defined  description  of  a  proposed 
system,  availability  of  detailed  bills  of  material,  detailed 
operations,  and  specialized  judgment.  The  method  produces  good 
results  for  systems  involving  standard  components  and  no  high 
risk  developments. 

ESSENTIAL  ELEMENT  OF  ANALYSIS:  A  question  specifically  designed  to 
obtain  data  that  will  provide  an  answer  in  a  particular  problem 
area,  or  information  required  to  conduct  an  evaluation  in  a 
particular  functional  area. 

EXPECTED  VALUE:  The  probability  of  an  event  occurring  multiplied  by 
the  payoff  associated  with  its  occurrence. 

EXTRAPOLATE:  Estimate  by  trend  projection  the  unknown  values  that  lie 
beyond  the  range  of  known  values  in  a  series. 

FAILURE  RATE:  The  number  of  items  replaced  per  unit  time  due  to  failure 
of  that  item. 

FEASIBILITY  STUDY:  (1)  A  study  of  the  applicability  or  desirability  of 
any  management  or  procedural  system  from  the  standpoint  of 
advantages  versus  disadvantages  in  any  given  case;  (2)  also  a 
study  to  determine  the  time  at  which  it  would  be  practicable  or 
desirable  to  install  such  a  system  when  determined  to  be 
advantageous;  (3)  a  study  to  determine  whether  a  plan  is  capable 
of  being  accomplished  successfully. 

FISCAL  POLICY:  Federal  Government  economic  stabilization  policies 

designed  to  foster  economic  goals  such  as  high  employment,  stable 
growth  and  prices,  and  balance  of  payments  equilibrium,  through 
changes  in  taxes  and  levels  of  Government  spending  as  distinct  from 
monetary  policy  (q.v,). 
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FIXED  COSTS:  Those  elements  of  cost  that  do  not  vary  with  volume  of 
production. 

FLOW  CHART:  A  graphic  representation  of  the  major  steps  of  work  in 
a  process.  The  illustrative  symbols  may  represent  documents, 
machines,  or  actions  taken  during  the  process.  The  area  of 
concentration  is  on  where  or  who  does  what  rather  than  on  how 
it  is  to  be  done. 

^  '  * 

FORCE  STRUCTURE  ANALYSIS:  The  analysis  of  proposed  forces  to  obtain 
a  pictyre  of  resource  implications  for  planning. 

'  *  .  \ 

FORCE  STRUCTURE  COSTING:  The  determination  of  the  resource  implications 
(manpower,  materiel,  support,  training,  etc.)  in  dollar  terms  of 
a  given  force  structure  or  change  to  it. 

HISTORICAL  COST:  The  cost  of  any  objective  based  upon  actual  dollar 
or  equivalent  outlay  ascertained  after  the  fact.  May  use  any 
of  many  methods  of  cost  determination. 

INCREMENTAL  COST:  The  added  costs  of  a  change  in  the  level  or  nature 
of  activity.  They  can  refer  to  any  kind  of  change:  adding  a  new 
product,  changing  distribution  channels,  adding  new  machinery. 
Although  they  are  sometimes  interpreted  to  be  the  same  as  marginal 
cost,  the  latter  has  a  much  more  limited  meaning,  referring  to 
cost  of  an  added  unit  of  output. 

INFLATION:  A  rise  in  the  general  level  of  prices.  (Pure  inflation  is 
defined  as  a  rise  in  the  general  level  of  prices  unaccompanied  by 
a  rise  in  output. 

INFORMATION  SYSTEM:  A  combination  of  personnel,  efforts,  forms, 

formats,  instructions,  procedures,  data,  communication  facilities 
and  equipment  that  provides  an  organized  and  interconnected  means-- 
automated,  manual,  or  a  combination  of  these--for  recording, 
collecting,  processing,  transmitting,  and  displaying  information 
in  support  of  specific  functions. 

INPUT-OUTPUT  ANALYSIS:  A  quantitative  study  of  the  interdependence  of 
a  group  of  activities  based  on  the  relationship  between  inputs 
and  outputs  of  the  activities.  The  basic  tool  of  analysis  is  a 
square  input-output  table,  interaction  model,  for  a  given  period 
that  shows  simultaneously  for  each  activity  the  value  of  inputs 
and  outputs,  as  well  as  the  value  of  transactions  within  each 
activity  itself.  It  has  been  applied  to  the  economy  and  the 
"industries"  into  which  the  economy  can  be  divided. 
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INVESTMENT  COST:  The  cost  beyond  the  Research  and  Development  phaise 
to  introduce  a  new  capability  into  operational  use. 

IAPIACE  CRITERION:  This  criterion- bases  decisions  under  uncertainty 
upon  the  assumption  that,  because  the  probabilities  of  future 
states  of  nature  are  unknown,  they  should  be  considered  to  be 
equal.  If  a  decision  is  based  on  the  Laplace  criterion,  equal  , 
probabilities  are  assigned  to  each  possible  state  of  nature,  ana 
the  alternative  which  maximizes  expected  value  if  selected.  The 
Laplace  criterion  is  sometimes  called  the  "Principle  of  Insufficient 
Reason. " 

LEARNING  CURVE:  The  cost-quantity  relationships  for  estimating  costs 
of  equipment.  Generally  used  to  predict  or  describe  the  decrease 
in  the  cost  of  a  unit  as  the  number  of  units  produced  increases. 

LEAST-SQUARES  METHOD:  A  method  of  fitting  a  calculated  trend  to 
statistical  data,  so  called  because  the  sum  of  the  squared 
deviations  of  the  calculated  from  the  observed  variables  is  a 
minimum.  "Least  squares"  also  refers  to  the  criterion  that,  when 
followed,  yields  this  result. 

MARGINAL  COST:  REVENUE:  Costs  incurred  or  expected  to  be  incurred  in 

the  production  of  an  additional  unit  of  output.  Marginal  revenue 
is  revenue  received  or  expected  to  be  received  from  the  sale  of 
an  additional  unit  of  output.  To  maximize  its  profits,  a  firm  has 
to  extend  production  to  the  point  where  marginal  revenue  equals 
marginal  cost. 

LOGARITHMIC  SCALE:  When  the  vertical  axis  of  a  chart  is  laid  off  in 
terms  of  the  logarithms  of  natural  numbers  the  arrangement  is 
known  as  a  semilog  chart  and  the  vertical  scale  is  called  a  log 
scale.  A  curve  plotted  on  such  a  chart  represents  not  the  numbers 
in  the  series  but  the  logarithms  of  these  numbers.  Changes  in  the 
slope  of  s'ach  a  curve  show  changes  in  the  percentage  increase  or 
decrease  of. the  original  series.  As  long  as  there  is  no  change 
in  direction,  equal  distances  on  the  vertical  scale  correspond  to 
the  same  percentage  change  in  the  original  series. 

MATRIX:  A  rectangular  array  of  terms  called  elements.  It  is  used  to 
facilitate  the  study  of  problems  in  which  the  relation  between 
these  elements  is  fundamental.  A  matrix  is  usually  capable  of 
being  subject  to  a  mathematical  operation  by  means  of  an  operator 
or  another  matrix  according  to  prescribed  rules. 


C-9 


MAXIMAX  CRITERION:  This  criterion  bases  decisions  under  uncertainty 
upon  the  optimistic  hypothesis  that  the  decision  maker  should 
select  that  strategy  which  produces  the  maximum  of  the  maxima. 
See  also:  Hurwiez  criterion;  Maximin  criterion. 

MAXIMIN  CRITERION:  (WALD  CRITERION)  This  criterion  bases  decisions 
under  uncertainty  upon  the  conservative  hypothesis  that  the 
alternative  which  produces  the  maximum  of  the  minimum  returns 
should  be  selected.  This  criterion  can  also  be  employed  (as 
can  the  Laplace,  et.  al.,  criteria)  in  minimizing  as  well  as 
maximizing  problems.  In  minimizing  problems,  one  seeks  the 
alternative  which  gives  the  minimum  of  all  maxima.  The 
criterion  in  these  circumstances  is  called  minimax  rather 
than  maximin.  See  also:  Hurwiez  criterion;  Maximax  criterion. 

MEAN:  The  most  common  measure  of  central  tendency  equal  to  the  sum 
of  the  observed  quantities  divided  by  the  number  of  observed 
quantities. 

MODEL:  A  simplified  representation  of  an  operation,  containing  only 
those  aspects  of  primary  importance  to  the  problem  under  study. 
The  means  of  representation  may  vary  from  a  set  of  mathematical 
equations  or  a  computer  program  to  a  purely  verbal  description 
of  the  situation.  In  cost/eff ectiveness  analysis  (or  any 
analysis  of  choice),  the  role  of  the  model  is  to  predict  the 
costs  that  each  alternative  would  incur  and  the  extent  to  which 
each  would  attain  the  objective. 

MONTE  CARLO  METHOD:  Any  procedure  that  involves  statistical  sampling 
techniques  from  a  distribution  of  possible  outcomes  for 
obtaining  a  probabilistic  approximation  to  the  solution  of  a 
mathematical  or  physical  problem.  Monte  Carlo  Methods  are  often 
used  when  a  great  number  of  variables  are  present,  with  inter¬ 
relationships  so  extremely  complex  as  to  forestall  straight¬ 
forward  analytical  handling.  This  method  generally  involves 
the  use  of  simulated  data  acquired  by  putting  random  numbers 
through  transformations  such  that  the  data  imitates  significant 
aspects  of  a  situation. 

NEW  OBLIGATIONAL  AUTHORITY  (NOA):  The  total  of  all  budget 

authorizations,  provided  by  current  or  prior  actions  of  the 
Congress,  of  whatever  type,  of  a  given  year  which  provide  new 
authority  to  incur  obligations.  NOA  has  consisted  of 
appropriation,  contract  authority,  or  authority  to  spend  debt 
receipts.  Under  the  Budget  Commission's  recommendations,  the 
word  "appropriations"  will  be  applied  to  this  concept  of  NOA. 


\ 
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OBLIGATIONS:  Obligations  in  Federal  accounting  represent  commitments 
to  acquire  materials  or  services  or  to  make  payments  under 
certain  conditions  (such  as  loans,  grants,  subsidies,  and 
contributions).  The  Congress  has  specifically  prescribed  the 
kinds  of  transactions  that  may  be  recorded  and  reported  as 
obligations  of  the  Government  of  the  United  States.  Total 
obligations  incurred  would  thus  be  the  amounts  of  orders,  places, 
contracts  awarded,  services  received,  and  similar  transactions 
requiring  disbursement  of  money.  In  contrast,  under  the  accrual 
basis  of  accounting,  expenditures  represent  the  receipt  of  funds, 
property,  or  services  within  a  given  period  of  time.  See: 

Accrual  accounting. 

OPERATING  COST:  The  recurring  cost  required  to  operate  and  maintain 
an  operational  capability. 

OPERATING  PROGRAM:  Conceptually,  a  mix  of  activities  and  resources 
under  common  management  which  represents  the  most  detailed 
organizational  or  budgetary  level  whose  identification  is  required 
in  the  information  system.  The  operating  program  may  be  identical 
with  a  program  element  (q.v.)  if  its  purpose  can  be  identified  by 
only  one  program  category  (q.v.).  Alternatively,  the  operating 
program  may  serve  more  than  one  such  purpose,  in  which  case  each 
part  of  the  operating  program  identified  by  a  discrete  program 
category  is  a  program  element. 

OPERATIONS  RESEARCH:  The  use  of  analytic  methods  adopted  from 

mathematics  for  solving  operational  problems.  The  objective  is 
to  provide  management  with  a  more  logical  basis  for  making  sound 
predictions  and  decisions.  Among  the  common  scientific  techniques 
used  in  operations  research  are  mathematical  programming,  statistical 
theory,  information  theory,  game  theory,  monte  carlo  methods,  and 
queueing  theory. 

OPTIMIZATION:  The  attainment  of  the  best  possible  result,  i.e.,  the 
maximization  (minimization)  of  some  desirable  (undesirable) 
criterion  measure,  subject  to  the  constraints  imposed  on  the  choice 
of  solutions. 

PARAMETER:  A  constant  or  a  variable  in  mathematics  which  remains 
constant  during  some  calculation.  It  is  generally  a  definable 
characteristic  of  an  item,  device,  or  system. 

PLANNING:  The  selection  of  courses  of  action  through  a  systematic 
consideration  of  alternatives  in  order  to  attain  organizational 
objectives. 
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PLANNING-PROGRAMMING- BUDGETING:  The  Planning-Programming-Budgeting 
System  (PPBS)  represents  an  effort  by  the  Executive  Branch  of 
the  Federal  Government  to  prepare  budgets  in  such  a  systematic 
way  as  to  make  them  most  useful  in  establishing  priorities,  in 
forward  planning,  in  choosing  between  programs,  and  in  measuring 
costs  against  meaningful  performance  yardsticks. 

POINT  ESTIMATE:  An  estimate  which  states  that  the  characteristic  of 
interest  has  a  single,  specific  value.  It  is  the  single  best 
estimate  of  a  population  value  made  from  a  sample.  This  may 
be  contrasted  with  an  interval  estimate  which  states  that  the 
population  value  of  interest  falls  somewhere  within  a  range  or 
interval . 

POPULATION:  If  a  set  of  data  consists  of  all  conceivably  possible 

(or  hypothetically  possible)  observations  of  a  certain  phenomenon, 
we  refer  to  it  as  a  population;  if  it  contains  only  part  of  these 
observations,  we  refer  to  it  as  a  sample. 

PRESENT  VALUE:  The  estimated  present  worth  of  a  system  of  future 
benefits  or  costs  arrived  at  by  discounting  the  future  values, 
using  an  appropriate  interest  rate. 

PROBABILITY:  A  number  between  0  and  1  that,  when  assigend  to  an 
event  or  occurrence,  expresses  the  likelihood  that  the  event 
will  occur. 

PROGRAM:  (1)  A  plan  or  scheme  of  action  designed  for  the  accomplish¬ 
ment  of  a  definite  objective  that  is  specific  as  to  the  time¬ 
phasing  of  work  to  be  done  and  the  means  proposed  for  its 
accomplishment,  particularly  in  quantitative  terms,  with  respect 
to  manpower,  material,  and  facilities  requirements;  thus,  a 
program  provides  a  basis  for  budgeting;  (2)  a  segment  or 
element  of  a  complete  plan;  (3)  a  budget  account  classification. 

PROGRAM  ELEMENT:  A  subdivision  of  a  program  category  (q.v.)  which 
comprises  the  specific  products  that  contribute  to  an  agency's 
objective(s).  If  an  agency's  operating  program  (q.v.)  is 
distributed  over. several  program  categories  (q.v.)  each  part 
of  the  operating  program  identified  by  a  discrete  program 
category  is  a  program  element. 

RANDOM  NUMBERS:  A  sequence  of  digits  in  which  each  digit  has  an 
equal  probability  of  occurring  in  each  position,  wholly 
independent  of  which  digits  appear  elsewhere  in  the  sequence. 
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RANDOM  SAMPLE:  A  sample  selected,  from  a  population  to  be  tested,  in 
such  a  manner  that  every  element  in  the  population  has  an  equal 
chance  of  being  chosen  for  the  sample. 

REGRESSION  ANALYSIS:  The  association  of  one  or  more  independent 

variables  with  a  dependent  variable.  Under  static  conditions  the 
analysis  is  called  correlation.  When  used  for  predictive  purposes, 
it  is  referred  to  as  regression.  The  relationships  are  associative 
only;  causative  inferences  are  added  subjectively  by  the  analysts. 


Where  only  one  independent  variable  is  used  the  technique  is  known 
as  simple  regression  and  takes  the  form  of  a  regression  line  (q.v.) 
of  the  equation: 

y  =  a  +  bx 

When  more  variables  are  added,  a  dimension  is  added  with 
variables,  making  graphical  representation  impossible  beyond  two 
independent  variables.  However,  the  general  analysis  is  the  same 
for  multiple  regression,  taking  the  general  equation  form:' 


y  =  a  +  bx.  +  cx  +  dx_  +  .  .  .  +  zx 
1  2  3  n 

For  effective  regression  analysis  to  be  undertaken,  the  data  base 
must  meet  certain  requirements. 


REGRESSION  LINE:  A  least  squares  equation  meeting  these  assumptions: 
a  stochastic  (q.v.)  relationship  in  which  the  random  error  term 
is  from  a  normally  distributed  population  with  zero  mean  and 
constant  variance.  See  also:  Regression  analysis. 


RESEARCH  AND  DEVELOPMENT  (R&D):  Basic  and  applied  research  in  the 
sciences  and  engineering,  and  the  design  and  development  of 
prototypes  and  processes.  Excludes  routine  product  testing, 
market  research,  sales  promotion,  sales  service,  and  other  non- 
technological  activities  or  technical  services. 

Basic  research  includes  original  investigations  for  the  advancement 
of  scientific  knowledge  that  do  not  have  specific  practical 
objectives. 


Applied  research  is  the  practical  application  of  knowledge,  material 
and/or  techniques  toward  a  solution  to  an  existent  or  anticipated 
military  or  technological  requirement. 


C-13 


Development  includes  technical  activities  of  a  nonroutine 
nature  concerned  with  translating  research  findings  or  other 
scientific  knowledge  into  products  or  processes.  Development 
does  not  include  routine  technical  services  or  other  activities 
excluded  from  the  above  definition  of  research  and  development. 

RESEARCH  AND  DEVELOPMENT  (R&D)  COSTS:  The  cost  of  developing  a  new 
capability  to  the  point  where  it  is  ready  for  procurement  for 
operational  units. 

RISK:  As  used  in  cost-effectiveness  analysis  and  operations 

research,  a  situation  is  characterized  as  risk  if  it  is  possible 
to  describe  all  possible  outcomes  and  to  assign  meaningful 
objective  numerical  probability  weights  to  each  one.  For 
example,  an  action  might  lead  to  this  risky  outcome:  a  reward 
of  $10  if  a  "fair"  coin  comes  up  heads,  and  a  loss  of  $5  if  it 
comes  up  tails.  Another  example,  50%  of  all  missiles  fired 
can  be  expected  to  land  within  one  CEP  of  the  target. 

SENSITIVITY  ANALYSIS:  Repetition  of  an  analysis  with  different 
quantitative  values  for  cost  or  operational  assumptions  or 
estimates  such  as  hit-kill  probabilities,  activity  rates,  or 
R&D  costs,  in  order  to  determine  their  effects  for  the 
purposes  of  comparison  with  the  results  of  the  basic  analysis. 

If  a  small  change  in  an  assumption  results  in  a  proportionately 
or  greater  change  in  the  results,  then  the  results  are  said 
to  be  sensitive  to  that  assumption  or  parameter. 

SETS:  A  collection  of  items  (elements)  chosen  as  pertinent. 

SIMULATION:  The  representation  of  physical  systems  and  phenomena 
by  computers,  models,  or  other  equipment.  The  model  or 
computer  representation  is  manipulated  to  imitate  significant 
aspects  of  a  situation. 

STANDARD  DEVIATION:  A  measure  of  the  dispersion  of  observed  data. 
Mathematically,  it  is  the  positive  square  root  of  the  variance. 

STANDARD  ERROR:  The  standard  deviation  of  a  group  of  measures  of 
the  same  characteristics  (often  termed  a  "statistic"  or  a 
"parameter")  each  obtained  from  a  distinct  sample  drawn  from 
a  larger  "universe"  or  "population". 

STATISTICAL  BIAS:  If  some  samples  or  observation  data  are  more 
likely  to  be  chosen  than  other,  or  if  subjective  methods 
are  used  in  selecting  sample  data,  the  results  are  considered 
biased. 


STATISTICAL  DECISION  THEORY:  Theory  dealing  with  logical  analysis 
of  choice  among  courses  of  action  when  (1)  the  consequence  of 
any  course  of  action  will  depend  upon  the  "state  of  the  world," 

(2)  the  true  state  is  as  yet  unknown,  but  (3)  it  is  possible  at 
a  cost  to  obtain  additional  information  about  the  state. 

STATISTICAL  INFERENCE:  Using  information  contained  in  a  sample  to 
make  predictions  about  a  larger  set,  the  population. 

STATISTICAL  METHOD  OF  COST  ESTIMATING:  A  method  of  cost  estimating 
utilizing  statistically  determined  cost  estimating  relationships 
which  express  cost  as  a  function  of  the  characteristics  specified 
for  the  case  in  question.  A  valuable  aspect  of  statistical 
estimating  is  that  an  objective  statement  regarding  cost 
uncertainty  can  be  provided. 

SUNK  COSTS:  Costs  which  are  irrevocably  committed  to  a  project;  such 
costs  normally  have  no  bearing  on  current  management  decisions. 

SYSTEMS  ANALYSIS  (SA):  A  formal  inquiry  intended  to  advise  a  decision 
maker  on  the  policy  choices  involved  in  a  major  decision.  In  DOD 
a  systems  analysis  may  be  concerned  with  such  matters  as  weapon 
development,  force  posture  design,  or  the  determination  of  strategic 
objectives.  To  quality  as  a  system  analysis  a  study  must  look  at 
an  entire  problem  as  a  whole.  Characteristically,  it  will  involve 
a  systematic  investigation  of  the  decision  maker's  objectives  and 
of  the  relevant  criteria;  a  comparison--quantitative  when  possible-- 
of  the  costs,  effectiveness,  and  risks  associated  with  the 
alternative  policies  or  strategies  for  achieving  each  objective; 
and  japu-attempt  to  formulate  additional  alternatives  if  thos’e'->x^ 
examined  are  deficient.  ^ 

.  *’ 

TIME-PHASED  COSTS:  A  presentation  of  the  cost  results  broken  down  by 
the  time  period  in  which  the  costs  occur  rather  than  a  single 
total  cost  figure. 

TOTAL  OBLIGATION  AUTHORITY  (TOA):  The  cost  allocated  to  a  given  system 
or  organization.  This  cost  when  related  to  a  specific  time  period, 
for  example,  a  year  represents  obligations  that  can  be  incurred 
during  that  year  and  not  necessarily  expenditures.  The  total 
obligation  authority  for  a  specific  year  to  furnish  a  house  is  the 
cost  of  what  can  be  ordered  during  that  year  even  if  deliveries 
and  payments  are  made  in  later  years. 
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TOTAL  SYSTEM  COST:  The  total  R&D,  Investment,  and  Operating  Costs 
(for  a  specified  number  of  years  of  operation)  required  to 
develop,  procure,  and  operate  the  particular  weapon  system, 

UNCERTAINTY:  A  situation  is  uncertain  if  there  is  no  objective 
basis  for  assigning  numerical  probability  weights  to  the 
different  possible  outcomes  or  there  is  no  way  to  describe 
the  possible  outcomes.  For  example,  the  probability  of  a 
foreign  nation  continuing  to  furnish  the  U.S.  with  base  rights 
is  an  uncertainty. 

VALIDATED  COST  DATA:  Resource  data  which  have  been  objectively 

analyzed  and  documented  by  the  preparing  agency,  and  independently 
evaluated  and  coordinated  with  those  Department  of  the  Army 
agencies  with  a  functional  responsibility  for  the  data. 

VARIABLE  COSTS:  Those  costs  that  vary  with  the  volume  of  output 

as  contrasted  with  fixed  costs,  which  do  not  vary  with  output. 

VARIABLES:  General  numbers,  such  as  x  or  y  which  may  take  on  many 
values  or  which  may  have  conditional  fixed  values  as  in 
x2  +  2x  =  19. 

WORK  BREAKDOWN  STRUCTURE:  A  standardized  division  of  a  weapon/ 
support  system  into  components  and  subcomponents.  This  is 
represented  by  a  structure  displaying  the  system  in  levels  of 
subcomponents  which  can  be  consolidated  into  higher  levels  of 
component  aggregation. 
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